The other day I had that big brained idea to play sound effects from code in Godot instead of playing some files.
I did not want to go download sound files for the different menu buttons bleeps, so I wrote some code to do it all in a little function.
My question is if there is anything wrong with this approach: does it create unnecessary load or is it just fine even for playing more sound effects? Would this approach scale?
What do you consider the pros and cons regarding the in-code approach?
The code below would create a bleep sound for something like play_tone(self, 400, 0.1, 10). In that case the for loop that calls _make_sample() would run about 4000 times or so.
extends RefCounted
const SAMPLE_RATE: float = 44100.0
static func play_tone(
parent: Node,
frequency: float,
duration: float,
volume: float,
) -> void:
var player: AudioStreamPlayer = AudioStreamPlayer.new()
player.name = "GeneratedSFX"
var generator: AudioStreamGenerator = AudioStreamGenerator.new()
generator.mix_rate = SAMPLE_RATE
generator.buffer_length = max(duration + 0.05, 0.1)
player.stream = generator
player.volume_db = 0.0
parent.add_child(player)
player.play()
var playback: AudioStreamPlayback = player.get_stream_playback()
if playback == null:
player.queue_free()
return
var sample_count: int = int(SAMPLE_RATE * duration)
var buffer: PackedVector2Array = PackedVector2Array()
buffer.resize(sample_count)
for i in range(sample_count):
var t: float = float(i) / SAMPLE_RATE
var phase: float = (TAU * frequency * t)
var wave: float = sin(phase)
var attack_time: float = min(0.01, duration * 0.2)
var attack: float = 1.0
if t < attack_time:
attack = t / attack_time
var decay: float = 1.0 - (t / duration)
decay = max(decay,0.0)
decay = decay * decay # curved decay.
var envelope: float = attack * decay
var sample: float = (wave * envelope * volume)
buffer[i] = Vector2(sample, sample)
playback.push_buffer(buffer)
var cleanup_time: float = duration + 0.05
_cleanup_player(player, cleanup_time)
That’s the big con, generating a sound sample isn’t particularly difficult, but you have to make at least 44100 samples per second. This makes it one of the hottest code paths in your project, you have to be very certain it’s as performant as possible.
Given your sample I’d say an easy improvement on the table is avoiding the function call _make_sample and pasting your function code directly into the for-loop body, GDScript has a pretty high overhead for function calls.
That’s a good point. I updated the snippet above to do it all in one go.
That’s true! Of course creating these samples can be done at a different time from playing the sound.
I want to point this out again, because it might not be clear from the code snippet: If one was to add a little timeout to the loop this would still create the same sound, just the creation would take a few seconds longer.
for i in range(sample_count):
await parent.get_tree().create_timer(0.0001).timeout # <- the timer.
var t: float = float(i) / SAMPLE_RATE
# ... more stuff
var envelope: float = attack * decay
var sample: float = (wave * envelope * volume)
buffer[i] = Vector2(sample, sample)
playback.push_buffer(buffer)
Even if you want to synthesize your own sound effects, you can get huge performance savings by saving the synthesized sound effects as files so you don’t need to don’t need to re-synthesize them any time you play them. If you can’t do that because the sound effect should be subtly different each time you play it, you can get huge performance savings by using GDNative (or, failing that, C#) instead of GDScript to synthesize your sound effect.
This got me wondering: if the sound effects can be generated ahead of time, why not do the whole synthesis process in the editor and avoid the runtime cost entirely?
For the above example I could load the Vector2(sample, sample) (of length 4000 per sound effect) already from disk without synthesizing it anew each time. So I would swap the loop for the load time of the file.
If one would be able to change the sound envelop in the editor with a simple editor plugin, I guess one wouldn’t even need to use GDNative or other languages. One could just do the synthesis part during editing or compile time.
I know there is plugins like that which port jsfxr or the inspiring original sfxr to Godot: gdfxr is one of them. I think this is also a pure GDScript approach without any performance problems.
I wonder if one could skip over the hard drive and just edit the sound in the editor, persist it without writing a file and still not suffering a performance boost during game runtime.
That is very scary code, makes your sample generation take much longer and if you only want to wait a frame then using await get_tree().process_frame is better, but if you are worried about frame stutters then threads are going to be the best option, and as others have mentioned using native code with GDExtension
If you do not save the samples to disk then it will not persist from editor to exported game. Storing your samples as a Resource, like other AudioStreams, will do you plenty good and if you extend AudioStream then it can be used with regular audio stream player nodes.
Sorry for that, I might not have been very clear about my intention here. I wanted to show that even if the building of the audio is interrupted the buffer will still contain the same samples and sound the same.
I should figure out how to put that data into AutoStreams and how to save Resources. I was reading already that Resources are good to persist data, but I couldn’t quite figure out how to do that.
Is there a way to get a AudioStream from a AudioStreamPlayback? I assume AudioStreamPlayback represents the runtime state, I guess the other way around would be easier.
I wouldn’t call this a big brained idea. This entire thing is going to turn into a huge mess, especially if the project gets bigger. The effort of generating these sounds at runtime, alongside introducing a large amount of code bloat which is going to be a pain to maintain and for future programmers to understand when the alternative is to play a 10kb mp3.
To me this is being way over complicated. There wouldn’t be any need to download any sound files, because they would be included with your project.
I don’t think it is ever possible to replace audio files entirely, like music and longer samples, but I think there could be a more useful tool to create some menu button sounds and such.
To be fair I meant that with a bit of irony. I’m not sure yet how good the ideas is or if there is any useful to it.
That could be extracted into a module or a plugin. I don’t see a problem there.
Exactly, the advantage would be not having to download files. Binary files would also not pollute the Git repo if everything could be handled in code.
There is an obvious advantage to tools like Sfxr (and Jsfxr and such) and that is that one can change each sound to one’s liking. There is an additional advantage of having such a tool in the Godot editor like Gdsfxr. (All links to these in my post above).
But these tools still generate a binary sound file which one stores on the disk. I wonder if there is a way to have an in-editor plugin to manipulate envelops of each sound and store them in text or Resources as discussed above without creating binary files. I think this could still reduce the overall complexity of a project, because there is never any sound files being created.
I wonder if that is possible technically in a way that is not harming performance.
Maybe, if it comes to more sophisticated sound management one would buffer the function calls (or could manipulate even the sample buffers) instead of managing audio files. At the other hand there is probably already good tools for that.