speech recognition using C# and System.Speech in godot 4

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By Pifpif

I’m trying to implement speech to text in my godot project. I couldn’t find any add ons for godot 4 so Im trying to do it myself using the system.speech nuget in VS code (C#).

I get an error “SAPI does not implement phonetic alphabet selection.” but from my research i dont think thats the problem.

My mic works, as I have tried using the audio player to play what comes from my mic. and i’ve tried the code seperate from godot and it worked even with the SAPI error.

Anyone know how to get this code to work, or know of another way to integrate offline speech recognition into godot 4?

Heres my code:

using Godot;
using System;
using System.Speech.Recognition;


public partial class AudioStreamPlayer : Godot.AudioStreamPlayer
{
	public override void _Ready()
	{
		GD.Print("START");
	
	}
	
    public override void _Input(InputEvent @event)
    {
        if (@event.IsActionPressed("talk")){

			using (SpeechRecognitionEngine recognitionEngine =new SpeechRecognitionEngine(new System.Globalization.CultureInfo("en-US")))
			{

				Choices command = new Choices();

				command.Add(new string[] { "hello" });

				GrammarBuilder grammarBuilder = new GrammarBuilder();
				grammarBuilder.Append(command);
				grammarBuilder.Culture = new System.Globalization.CultureInfo("en-US");

				Grammar grammar = new Grammar(grammarBuilder);
				recognitionEngine.LoadGrammar(grammar);


				recognitionEngine.SpeechRecognized += new EventHandler<SpeechRecognizedEventArgs>(RecognitionEngine_SpeechRecognized);


				recognitionEngine.SetInputToDefaultAudioDevice();

				recognitionEngine.RecognizeAsync(RecognizeMode.Multiple);

			}

		}

		if(@event.IsActionReleased("talk")){
			GD.Print("release");
		}
    }


	private static void RecognitionEngine_SpeechRecognized(object sender, SpeechRecognizedEventArgs e)
	{
		GD.Print("____________________________________________________");
		GD.Print("test recognized");
		GD.Print(e.Result.Text + "\n");

    }

}

Edited to fix forum code formatting.

jgodfrey | 2023-04-14 23:33

Do you have a scoping issue here? That is, the using clause here is destroying the recognitionEngine instance when _Input is done. I suspect your example may have had a loop somewhere to keep it listening while running (e.g. like in a console).

Try setting up the speech engine in _Ready so to extend its lifetime and perhaps only subscribe to its events while input is held.

spaceyjase | 2023-04-16 10:13

yep that was it, along with having sonic studio installed… for some reason

Pifpif | 2023-04-18 11:37