Managing structure changes when saving a class as resource

[Lutzilutz]

Context

I’m currently developing an in-house software to manage a database of hardware. The database is a class, that is saved as a resource with

ResourceSaver.save(my_database, database_path)

This database is loaded with

load(database_path)

From within the database class. This class has only one variable, an array of “Gear” (hardware element if you prefer). The gear class contains multiple variable, with the export keyword so that it is saved.

Everything works like a charm, but I’m trying to build a software which handle any change in the structure (I mean changes in the number of variables in the Gear class, or their name, or type of variable).

What I want to achieve

I’m simplifying a bit. Let’s say my Gear class has only one variable :

@export var id: int

So my database contains an array of this single-variable Gear instance.

What happens if :

• I save the current database (single variable in Gear instances).
• I modify the software by adding another variable to the Gear class such as below
• I launch the software again, and load the resource database.

@export var id: int
@export var name: String

My question

Godot would expect the resource to contain 2 variables by Gear, but the saved resource only contains 1. What would happen ? Will Godot add a default value to this new variable, or throw an error, or something else ?

More generally, how should I handle a saved resource, if in the future I want to add more variables, or delete some, or rename ?

I would like the user experience to be hasslefree, any changes to the software should be able to still load a previous version of the database.

[mrcdk]

@Lutzilutz

Godot would expect the resource to contain 2 variables by Gear, but the saved resource only contains 1. What would happen ? Will Godot add a default value to this new variable, or throw an error, or something else ?

It will give it the default value it has. In the example, name would have an empty string.

@Lutzilutz

More generally, how should I handle a saved resource, if in the future I want to add more variables, or delete some, or rename ?

There’s no versioning mechanism. If you add a new variable it will use the default value, if you remove a variable it won’t load that variable and if you rename a variable it would be as if you removed the variable and added a new variable.

If you need more control serializing and de-serializing the data then using a Resource won’t be suitable. You may want to use something like SQLite like GitHub - 2shady4u/godot-sqlite: GDExtension wrapper for SQLite (Godot 4.x+)

[Lutzilutz]

Thank you, that clears a lot ! I couldn’t find anything about all that.

Would you say, it’s possible to manage these changes in code (maybe with the help of a version identifier in the resource class) ? Like exhaustively code what the software should do with previous versions. I have a hard time judging if it could work.

I think you already solved my question I’ll wait a few days before marking it solved, to see if someone has something major to add.

[mrcdk]

@Lutzilutz

Would you say, it’s possible to manage these changes in code (maybe with the help of a version identifier in the resource class) ? Like exhaustively code what the software should do with previous versions. I have a hard time judging if it could work.

Not possible out of the box. You could write a ResourceFormatLoader and ResourceFormatSaver and do the serializing and de-serializing of the data yourself if you still want to treat the file as a “Resource” inside the editor (use load()/preload()/ResourceLoader.load() to load it and ResourceSaver.save() to save it or use the file in the editor)

An example on a EditorPlugin implementing this:

@tool
extends EditorPlugin

var my_loader:MyLoader
var my_saver:MySaver

func _enter_tree() -> void:
	my_loader = MyLoader.new()
	ResourceLoader.add_resource_format_loader(my_loader)

	my_saver = MySaver.new()
	ResourceSaver.add_resource_format_saver(my_saver)



func _exit_tree() -> void:
	if my_loader:
		ResourceLoader.remove_resource_format_loader(my_loader)
		my_loader = null

	if my_saver:
		ResourceSaver.remove_resource_format_saver(my_saver)
		my_saver = null


class MyLoader extends ResourceFormatLoader:

	func _handles_type(type: StringName) -> bool:
		return type == &"Resource"

	func _get_recognized_extensions() -> PackedStringArray:
		return ["test"]

	func _load(path: String, original_path: String, use_sub_threads: bool, cache_mode: int) -> Variant:
		var file = FileAccess.get_file_as_string(path)
		var json = JSON.parse_string(file) as Dictionary

		var res = TestResource.new()
		res.id = json.get("id", 0)
		res.name = json.get("name", "")

		return res


class MySaver extends ResourceFormatSaver:

	func _get_recognized_extensions(resource: Resource) -> PackedStringArray:
		return ["test"] if resource is TestResource else []

	func _recognize(resource: Resource) -> bool:
		return resource is TestResource

	func _save(resource: Resource, path: String, flags: int) -> Error:
		resource = resource as TestResource
		var json = {
			"id": resource.id,
			"name": resource.name
		}
		var file = FileAccess.open(path, FileAccess.WRITE)

		if file:
			file.store_string(JSON.stringify(json))
			file.flush()
			return OK
		else:
			return ERR_FILE_CANT_WRITE

This will let you create files with the extension test that will be serialized to json and de-serealized from json.

It only saves two fields, the other two aren’t saved and if you modify those values they won’t be serialized so when loading the resource back those will have the default value.

This example may only work on the editor and not in a exported project. You can do the same in a autoload/singleton but I already had some code for a test plugin so I did it like this.

[gentlemanhal]

The comments in this thread might give you some ideas.

How do I clean up old versions of a game when updating to new version?

You could also save the local files into a separate folder for each version. If the folder for the latest version doesn’t exist, check the older version’s folder and migrate the data. Afterwards remove the older version’s folder entirely. But to be honest, you could achieve it in many different way and I believe you’ll find a convenient way to do it

[Lutzilutz]

Thanks mrcdk for the long reply and code example ! That’s gonna be precious for me. You’re amazing I have 2 small questions below.

@mrcdk

This will let you create files with the extension test that will be serialized to json

What is the motivation to not use json directly ? Is it to avoid users to be able to read the .test file ?

Initially I wanted to avoid using JSON (considering Godot is offering easy-to-use resources and saves). But I could live with that.

@mrcdk

@tool
extends EditorPlugin

Is the @tool keyword mandatory to work ? I haven’t really use it yet, if possible I’d like to remove it until I have a better understanding of it.

@gentlemanhal

The comments in this thread might give you some ideas.

Thanks to you, that’s gonna be useful too !

[mrcdk]

@Lutzilutz

What is the motivation to not use json directly ? Is it to avoid users to be able to read the .test file ?

Oh, you can use JSON directly if you want but then it wouldn’t behave like any other resource in the engine. The example was more to show that you can implement it in a way that will be transparent to the editor as it will act as any other resource. I just used JSON because it was simple to showcase. You can serialize it however you want.

@Lutzilutz

Is the @tool keyword mandatory to work ? I haven’t really use it yet, if possible I’d like to remove it until I have a better understanding of it.

Yes. If you choose to do it this way then you’ll need to register the ResourceFormatLoader and ResourceFormatSaver in the editor too so you’ll need to use a @tool script for that.

Again, you could do something like this and call it a day:

class_name TestResource extends Resource

@export var id:int = 0
@export var name:String = ""


func save_to_file(path:String) -> Error:
    var json = {
        "id": id,
        "name": name
    }
    var file = FileAccess.open(path, FileAccess.WRITE)

    if file:
        file.store_string(JSON.stringify(json))
        file.flush()
        return OK
    else:
        return ERR_FILE_CANT_WRITE
        
        
static func load_from_file(path:String) -> TestResource:
    var file = FileAccess.get_file_as_string(path)
    var json = JSON.parse_string(file) as Dictionary

    var res = TestResource.new()
    res.id = json.get("id", 0)
    res.name = json.get("name", "")

    return res

[Lutzilutz]

Thanks once again, this is really helpful ! I managed to build a basic working first draft, based on your second and simpler example . I’ll probably stick with it.