How to structure a large ammount of data for editing?

Help Programming
1

Godot Version

v4.6.stable.official [89cea1439]

Question

TLDR: How to improve Editor experience when defining a large mapping between ID and resource?

Recently, I’ve been working on a sound manager for godot, inspired by [resonate]) Widgit Gaming / Godot / Resonate · GitLab ).

The basic underlying idea is that you define all of your SFX inside of resources, including stuff like the stream(s) to use, pitch, volume and bus control. Then in your gameplay code, you refer to these sounds by ID (or rather, a combination of LibraryID:SoundID).

After getting quite far with my idea, I’ve come to the realization that neither of Godots exported containers are really that suitable for organizing this quantity of data.

Dictionaries

The obvious choice, but it has so many drawbacks. You cannot edit the keys of exported dictionaries, and you also cannot search or filter on dictionary keys. Actually editing all of your SFX this way would be a nightmare.

Arrays

If you inline the ID of the sound into a property on the resource, then technically you can define your sounds in a big array. But the discoverability here is even worse… re-finding a specific sound effect to edit will be nearly impossible unless you save all of them to disk.

Custom Property Editor

I’ve actually explored this, and it seems somewhat workable, but it’s so finicky to implement. The primary concern, is that I want users to be able to define their SFX as resources. And re-implementing the entire resource workflow just seems kinda tough. You can kinda get there with a combination of EditorInspector and EditorResourcePicker, but I just fear I would be building myself into a corner.

Any ideas?

Since it’s a plugin, I don’t neccesarily want users to have to use any other plugin to make my plugin usable. e.g., stuff like Edit Resources as Table 2 - Godot Asset Library . So I’m feeling a bit stick.

Suprisingly, all of the issues I have with the properties panel have open issues on Godot Proposals, but the activity has primarily died down. So I don’t think waiting on engine changes makes sense.

2

Spreadsheets. Can’t beat them for managing lots of data.

2 Likes
3

Is this only about editing or storage as well? Because for the latter I’d say SQLite. As for editing, yeah, tables are usually quite nice once you have lots of rows/datasets.

1 Like
4

If the data wouldn’t contain any references to Godot assets, I might agree. Just setting volume/pitch/bus in a spreadsheet makes a lot of sense.

But in this case, I also want to be organizing the sound(s) that will play when this event fires. I don’t like the idea of using an external godot tool to refer to godot resources.

5

What do you need to refer to? Use strings and let your scripts build internal structures from spreadsheet data at startup (or whenever something changes). You can check/report any errors there.

6

Strings being the name/path or I guess UUID of the imported sound files?

I guess that’s an approach I will need to consider. I had some plans for some editor integrations that only really make sense if the data lives as resources.

7

Depends on how it’s intended to be used. Names are more user friendly but uids are more bulletproof. Mistakes are possible either way but you can always catch them. Imo, it’s a very reasonable price to pay for not needing to mess with the inspector gui.

1 Like
8

Neat idea but I hate that they call them SoundManager and Music Manager. Hate it.

That sounds needlessly complex for users.

Dictionaries

Dictionaries seem like a bad choice.

Not sure what you mean by this. You can’t “edit” a Dictionary key ever. You can delete the old one and make a new one.

Yes you can.

Arrays

This sounds awful.

Custom Property Editor

Have you considered just using Resources as-is?

Other Ideas

Have you considered a UI similar to the AnimationPlayer? Or something like VisualShaders?

It sounds like you are trying to figure out a UI for your data, rather than a storage method for them, and that’s somehow getting conflated.

It might help if you were to show us what a sample Resource would hold with some code.

Ultimately, based on what I’ve read, you’d probably be better off making your own custom AudioStream or AudioStreamPlayer instead of trying to re-invent the wheel. A lot of the the functionality you seem to want to add already exists in AudioStreamGenerator, AudioStreamInteractive, and AudioStreamRandomizer.

9

Thanks for the detailed reply! You’re correct that i’m mostly focused on editing right now - the underlying storage format isn’t super important to me.

I think you’re probably correct that the best way of handling it would be defining the sound events as files (instead of inline in some container). Discovery is pretty easy then, because you can use the filesystem itself to search for them by name.

The biggest reason I didn’t do that, is there isn’t any fast or convenient way to get all resources of a specific type. I mean you can search the filesystem for them, but that’s not ideal.

There are already plenty of other ways to manage audio. Like using AudioStreamPlayers directly, or using something like SoundManager by NathaHoad.. This particular plugin I’m working on is attempting to solve a specific issue that I faced when doing gamejams with an audio engineer friend.

So really, the primary goal is to seperate the audio implementation from the gameplay code, so he can tweak values independently.

Just to add a bit more information:

The plugin adds a new resource called Sound with fields like this:

@tool

class_name Sound extends Resource

@export var name : String

@export var id : String = ID.v4()

@export var stream : AudioStream
@export var volume_offset_db : float
@export_range(-20, 20) var volume_offset_randomization : float
@export var pitch_offset_db : float
@export_range(-5, 5) var pitch_offset_randomization : float

func _validate_property(property: Dictionary) -> void:
	if property.name == "id":
		property.usage = PROPERTY_USAGE_READ_ONLY | PROPERTY_USAGE_EDITOR
	
func get_stream() -> AudioStream:
	return stream

Then I create children versions of this for stuff like randomized streams, weighted randomized streams, linear playlists, and probably polyphonic handling as well.

I was just searching for a convenient editor interface for him to work on these sounds. But you’re probably correct: Defining them as independent files probably is best.

1 Like
10

I also added a type of resource called SoundReference with an editor customization.

@export var click_sound : SoundReference

So if you want to refer to a sound, you can export this type, and the editor will give you a control for selecting the sound. This is inspired by e.g., Unreal Engines LOCTEXT implementation.

image

The play button automatically previews the sound, and I will also add some other controls there later, like an edit button that takes you directly to the sound resource.

Of course if you’ve exported a sound reference, you would be able to do SoundManager.play(click_sound) as well.

1 Like
11

Resources and spreadsheets are not mutually exclusive. That’s why I said to build internal data structures from the spreadsheet data. Those data structures would likely be resources, for the most part.

2 Likes
12

Now that I know you’re just doing this for one person, you should be asking him how he wants to interface with it. For someone who knows what they want, a spreadsheet can be perfect. You export it as a .csv, tell localization to ignore it, and all he does is copy the new version over into the filesystem. Then he either re-imports it, or you just read it at runtime.

I’d recommend starting low tech. But ask your end user.