How to manage circular dependencies in resources?

Godot Version

4.2.1

Question

Hi everyone, I ran into a wall in my project.

I need to have resources that references other resources.
Let’s get an example from the real world. The body needs to regulate the sugar in our blood. So we have a resource - “sugar” - that knows two other resources - “A” and “B”, two not nicely named molecules that act as messengers. When “sugar” is under a threshold, it emits “A”, that will increase the sugar in the blood. So, “A” is another resource, that references “sugar”. So does “B”, that acts the other way around. It’s called a feedback loop, and it’s like 60% of how our body maintains its balance.

And it seems to be impossible to have data that acts that way in Godot, while it’s possible in Unity.
I started the search with my scene being broken on load, to find a message about missing dependencies that yet still existed at that exact path, to blank references in play mode where I had filled the inspector before… And ended up on that post : (Allow circular dependencies/references for resources in export var · Issue #7363 · godotengine/godot-proposals · GitHub)

What they’re talking about is barely within my understanding, so if I say something inaccurate please correct me.
When Godot wants to load, it starts with a resource. When it finds in that resource another one, it jumps to that one, and will load the second one before the first, before going back to the first and load it too. The problem is, if the second resource references the first, then it goes back to the first, which leads to the second etc… In an unending loop. It solves this by breaking out of it and renouncing, leaving blank references. It avoids a full crash of the engine.
I read on the discussion something about RefCounted, weak links and other stuff… But it’s not really explained concretely how to do it, not at a step by step level that I can understand. So, here is finally my question :

How could I implement a feedback loop with data (Resources ? Something else ?) and not nodes, so it’s possible to edit it from a game designer perspective (aka in the inspector) to be maintainable in the long run ? I mean, in the body there are thousand of molecules and hundreds of indicator to keep track of, so I can’t really hard code anything and open the scripts every time I need to add something… Are there some of you that do it in your projects ? How do you do it ? How would you do it if you needed to ?

Thank you very much for reading that far !

Solving a problem like a dynamic system with many individual units, where the state of any unit can depend on the previous states of any of the other units (including itself) is a pretty complex and large topic. The way to build such a simulation can be very different depending on what the character of the things you are simulating is, and what kind of performance you are targeting, as well as which systems you are able to understand. I’ll try to give some general advice thoug.

Using Resources might not be the best path for what you are trying to do. Resources are a type of Object that specifically is able to be saved as a file, this is their primary purpose. So if a resource A contains another resource B within it, then saving A to a file first requires turning B into data that can be saved into a file, and then including that data along with the rest of what A needs in a file.

Since both A and B could be saved as files, B cannot possibly contain A, if it did then saving B would require collecting all the data required to save A, which includes all the data required to save B, which includes all the data required to save A… to infinity…

This problem has many different solutions, but you want something that will help you be able to edit the behavior of your simulated units in the inspector rather than changing scripts all the time, there aren’t any existing inspector that fit this very closely without having to do a little work, but lets try something like this:

I’ll assume that each unit has up to 4 internal values that it can keep track of for simplicity and it can “announce” any number of external values that other units will be able to see, but only on the following timestep, I’ll call the collection of all the announced values the “environment”. I’ll also assume that the new value of the unit’s internal values can be expressed as an “expression” which can be handled by the Expression class in godot. The same will apply to announcements which will be evaluated after the internal values are updated so that they can use those new values. Finally I’ll assume all values can be represented as floats so numbers including a fractional component like 3.529

The units now only need to store their internal values, the expressions to update them, and the expressions to announce new values. They will not directly store other units, so they can be easily represented as a Resource. The script will look something like this, but this is psuedocode so it’s just a general idea:

class_name SimUnit
extents Resource

@export var a: = 0
@export var b: = 0
@export var c: = 0
@export var d: = 0

const vals_index = ["a", "b", "c", "d"]

@export var update_a: String = ""
... b, c, d

@export var announcements: Array[String] = []

var update_expressions: Array[Expression] = []
var announce_expressions: Dictionary = {}

func _ready() -> void:
  # parse expressions here making Expression obj for each of them taking ["a", "b", "c", "d"] as variables
  update_expressions = [...parse(update_a)]
  announce_expressions = {announcements[i].split('<>')[0]: parse(announcemts[i].split('<>')[1]..., ...}

func do_timestep(old_environment: Dictionary) -> Dictionary:
  var my_values: = [a, b, c, d]
  for index in range(4):
    var abcd = vals_index[index] 
    var result = update_expressions[index].execute(my_values, old_environment)
    set(abcd, result)

  var new_announcements: = {}
  for an_name in announce_expressions:
    var result = announcement_expressions[an_name].execute(my_values, old_environment)
    new_announcements[an_name] = result

  return new_announcements

There would be a simulator script that runs all the units do_timestep passing them the dictionary of announcements, the expressions could read the announcement values like get('sugar_level') and use the internal values just with a or b etc.

In the inspector you’d write announcements like “name<>get(‘other’) + (a * 2)”
where “<>” or some other unique separator is used between the name and the expression, each internal value would have it’s own expression string

You could then save all the units into an array or resources in a node or other resource to keep track of them
If you wanted to have a heirarchical structure you could also add something like

@export var child_units: Array[SimUnit] = []

And then you could nest other units inside this one, it would purely be for organizational purposes though, they would still interact with announcements in the same way, the simulator would need to loop through all the child units recursively as well

Whenever any other part of the game or program wants to interact with the system it should also be able to tell the simulator to add an announcement or read out what the announcements were for the last frame.

This is just one example of how make something like this making use of godot’s built in editor to help make iterating on a design easier. In reality, the indidual project will almost always be much easier to work with if the tools are created with that project in mind, being able to make your own UI and editing tools, both using @tool scripts and by running an editor scene and saving Resource files is the main strength of godot here, since you can very quickly make an editor or tool that suites your exact needs once you figure out what they are, don’t shy away from making your own tools, but explore until you know something about what you need first. Good luck!

I’ve run into this before ( Problem with Recursion in combinable resources - #6 by intangiblematter )

My workaround was that I’ve changed my script to find the referenced resources by other means than assigning the resource directly in the inspector. Instead I mostly use certain properties and include type checks in the script. It works pretty well, but is of course heavily dependent on how your resources are set up and can break a bit easier.

@phazorknight
Thank you for your answer ! It will make things a little bit akward, but I guess I can copy-past the name I assigned to the resource I need at some point to break the loop, and get the resource back from a manager. Not elegant, not really maintainable, but it should be enough for my needs as a learning solo dev.

@cammymoop
I followed your explanations most of the way… Then got lost. Sorry. I guess I’ll have to keep getting experience before attempting some complicated changes like you suggest.
Tough you surprised me there : “Using Resources might not be the best path for what you are trying to do.” As far as I know, there are only Nodes and Resources in Godot ? The doc presents Resources as data container… And never implies there are other data containers. Is there another built in type ? Or can I make a type, like with a plain C# script, but in GdScript ?

Thank you anyway people for taking the time to read and answer !

A resource shouldnt know anything of anything else. It is supposed to be a data object. Data doesnt care what other data does.

To stick to your example: It is not the sugar that reports that something is of but the sugar is only the data. Some controlling function measures constantly if there is enough or too much of each molecule and sends signals to either stop production or increase it.

1 Like

True, it is the organ (in this case the pancreas) that measures the levels of sugar in the blood and sends the messengers. I was merely trying to reduce the example to a two components system.

Unfortunately, it doesn’t change the fact that I would want to have the organ be a Resource too, in this case, and have a more general purpose Node handle a list of organs and use the data they store (which molecule to call for example, and on which occasion) to conduct calculations and calls. Again, to make it more maintainable, I would just create an instance of the organ Resource for each organ that I have, put them in a list a be done. And if I need to add one later, I’d just create one in the same folder and let the magic solve my dependency injection with Godot Resource Groups add-on (GitHub - derkork/godot-resource-groups: An add-on for the Godot engine that simplifies handling of resources).

Or maybe is it that I misunderstand when I should use a Resource, and when I should use a Node ? I mean… I would use a Scriptable Object in Unity, which acts as data container, not being called by the normal game loop calls (update). I don’t need an in-scene object to do what I want, so I thought of Resources… Am I wrong ?

Plus, to answer directly : “Data doesnt care what other data does.” → I don’t need them to care what other data do. I need them to tell which data they should impact. They won’t communicate directly, I’ve managers for that that are hooked up to signals.
In the case of the organ (Pancreas) it doesn’t regulate only sugar. So, it should know of the sugar indicator it should check on, and the other ones. They would be resources. The sugar would then know what it’s states are, and if there are messengers to call. And each messenger should know what indicator they should impact. So everyone takes care of their own little responsibility, the manager doesn’t know about any data it shouldn’t, it just look at each resource, see if a condition is met, and if so calls the corresponding logic. The messenger don’t know who should call it, it doesn’t matter, it could even be used in different systems.

Using a Resource for your entities is fine, Resources can either be saved as a separate file or bundled into a scene file so they are a good way to represent persistent editable data.

Theoretically it would be fine for resources to reference each other in a loop as long as those resources are saved in separate files, because those files could save just the path to the other resource, but instead resources instead consider any properties it has which are also Resources to be “sub-resources” so even if the data is from a different file it considers it to be part of it’s hierarchy of data that it owns, which can’t work if there’s a circular relationship.

Instead of using a Resource property to reference another resource though, you can use a NodePath, which can include property paths “subnames”, so If a node that held resources was /root/Scene/ResourceHolder and it had a resource called spleen then get_node_and_resource("/root/Scene/ResourceHolder:spleen") can be used to get that resource, so the NodePath /root/Scene/ResourceHolder:spleen can be saved in one resource to reference another resource, it can also reference subresources of that resource like /root/Scene/ResourceHolder:heart:left_ventrical.

The catch here is I don’t think it supports indexing arrays, so you’d need to implement that yourself if you needed it. But the idea of using a path to a resource to reference it rather than using the other resource directly is the same.
You could probably implement it with a NodePath and array of indexes and property names so if you wanted a path like

/root/Some/Node:organs[2]:vessels[4]:membrane:wetness
You would need to save the part up to the first array indexing as the NodePath:
/root/Some/Node:organs
And then you’d have an array of strings like:
["[2]", "vessels", "[4]", "membrane", "wetness"]

so a hypothetical function to set a value using that format would be something like

func set_on_obj_arr(obj_arr, key: String, new_value) -> void:
  if key.begins_with('['):
    var index = int(key.trim_prefix('[').trim_suffix(']'))
    obj_arr[index] = new_value
  else:
    obj_arr.set(key, new_value)

func get_from_obj_arr(obj_arr, key: String) -> Variant:
  if key.begins_with('['):
    var index = int(key.trim_prefix('[').trim_suffix(']'))
    return obj_arr[index]
  else:
    return obj_arr.get(key)

func set_property_at_path(primary_path: NodePath, subpaths: Array[String], new_value) -> void:
  var got_path = get_node_and_resource(primary_path)

  # If there are no Resources in the NodePath part then use the Node to start with
  var obj = got_path[1] if got_path[1] != null else got_path[0]
  var get_props = [] if got_path[2] == null else str(got_path[2]).split(":")
  get_props.append_array(subpaths)
  final_prop = get_props.pop_back()
  
  for prop in get_props:
    obj = get_from_obj_arr(obj, prop)
  set_on_obj_arr(obj, final_prop, new_value)

var primary = ^"/root/Some/Node:organs"
var sub = ["[2]", "vessels", "[4]", "membrane", "wetness"]

# Update the membrane resource's wetness
set_property_at_path(primary, sub, 5.0)

(this code doesn’t implement getting or setting properties of various vector types or other non-objects that have properties, Dictionary keys do work like property names though)
Similar thing for getting the value from that kind of split path.

both primary and subpaths could be exported properties on any resources that need to point to another resource

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.