Problem with Hugging Face`s API response. Deadline - 25/02/25 (DD/MM/YY)

Godot Version

4.2.1

Question

So i have a line here that connects to the Hugging Faces API “Gemma 7b” model. Im making a school project where the player types in a input for a given survival scenario and AI tells the player if they will survive and if not - why. Think of Death by AI from Discord. Everything works great..but...there was one small issue which is - basically the players input is sent to the API but it doesnt return a response. I gave the code to ChatGPT and now it is hell…On the output part it gives out the players input instead of the APIs plus on the part where it is written “comment:” it stays blank (This should be the API`s response…

Here are the codes:

class_name GameStart
extends Node2D

var scenarios = ["Venomous snake attacks you", "Lost in a forest", "Stuck in a snowstorm", "Caught in a rip current", "Encounter a wild bear", "Trapped in a burning building", "Stranded on a deserted island", "Caught in an earthquake", "Witness someone choking", "Caught in a flash flood", "Stuck in an elevator", "Facing a home invasion", "Bitten by a spider", "Falling through ice", "Stuck in quicksand", "Power outage during a storm", "Vehicle breaks down in the desert", "Trapped in a cave", "Lost at sea", "Facing a tornado", "Attacked by a dog", "Poisoned by a plant", "Caught in a landslide", "Encounter a mountain lion", "Caught in a volcanic eruption", "Stranded in a swamp", "Witness a car accident", "Bitten by a tick", "Caught in a blizzard", "Stuck in a sinking car", "Encounter an aggressive moose", "Struck by lightning", "Lost in the mountains", "Facing a tsunami", "Struck by hail", "Attacked by a shark", "Caught in a wildfire", "Facing a nuclear explosion", "Encounter a swarm of bees", "Trapped in a mudslide", "Facing a power outage during extreme heat", "Stuck in a collapsing building", "Witness a drowning", "Encounter a rogue wave", "Lost in the desert", "Encounter an aggressive elk", "Facing a hurricane", "Poisoned by a chemical", "Attacked by a wild boar", "Encounter a tornado on water", "Stuck in a cave-in", "Facing a water shortage", "Attacked by a cougar", "Facing a sandstorm", "Stranded in the Arctic", "Encounter a wild buffalo", "Facing a blackout in a major city", "Encounter an electric eel", "Caught in a sinkhole", "Attacked by a large bird", "Facing a gas leak", "Stranded on a cliff", "Attacked by a piranha", "Caught in a snow avalanche", "Stuck in a maze", "Facing a medical emergency alone", "Encounter a rabid animal", "Poisoned by contaminated water", "Lost in an unfamiliar city", "Facing a heatwave without AC", "Witness a robbery", "Encounter a venomous lizard", "Stuck in a traffic jam during a crisis", "Facing a food shortage", "Attacked by a scorpion", "Encounter a wild horse", "Facing a blackout in freezing temperatures", "Stuck on a runaway train", "Attacked by a stingray", "Caught in a tidal wave", "Stuck in a dark forest at night", "Facing a bee sting allergic reaction", "Encounter a wild pig", "Facing a vehicle fire", "Attacked by a jellyfish", "Encounter a wild goat", "Stuck in an abandoned building", "Facing a severe allergic reaction", "Encounter a wild turkey", "Facing a sewage backup", "Stranded on a mountaintop", "Attacked by a lionfish", "Encounter a wild donkey", "Facing a broken bone in the wilderness", "Attacked by a fire ant", "Encounter a wild sheep", "Struck by a falling object", "Facing a ruptured water pipe", "Attacked by a centipede", "Encounter a wild yak"]

@onready var startB = $Start as Button
@onready var backtomenuB = $BackToMenu as Button 
@onready var nextscenarioB = $NextScenario as Button
@onready var api_token : String = "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"
@onready var API_URL : String = "https://api-inference.huggingface.co/models/google/gemma-7b"
@onready var temperature : float = 0.7
@onready var max_tokens : int = 100
@onready var headers = ["Content-Type: application/json", "Authorization: Bearer " + api_token]
@onready var request : HTTPRequest = HTTPRequest.new()

var callback_function: Callable

func _ready():
	randomize()
	var random_index = randi() % scenarios.size()
	var random_scenario = scenarios[random_index]
	add_child(request)
	request.request_completed.connect(_on_request_completed)
	
func chat_request(player_chat: String, callback_function: Callable, random_scenario: String):
	self.callback_function = callback_function
	var prompt_text ="Based on the following, provide a concise comment (30 words max) that evaluates the player's input logically. Do not repeat the scenario or the player input. Only output your comment.\n\nScenario: " + random_scenario + "\nPlayer Input: " + player_chat + "\nComment:"
	var body = {
		"inputs": JSON.stringify([{"role": "user", "content": prompt_text}]),
		"parameters": {
			"temperature": temperature,
			"max_tokens": max_tokens
			}
			}
	var json_body = JSON.stringify(body)
	var send_request = request.request(API_URL, headers, HTTPClient.METHOD_POST, json_body)
	if send_request != OK:
		print("Failed to send request! Error code: ", send_request)

func _on_request_completed(result, response_code, headers, body):
	var body_text = body.get_string_from_utf8()
	print("Response code: ", response_code)
	print("Raw Response body: ", body_text)
	if response_code == 200:
		var json_parser = JSON.new()
		var parse_result = json_parser.parse(body_text)
		if parse_result == OK:
			var response_data = json_parser.get_data()
			if typeof(response_data) == TYPE_ARRAY and response_data.size() > 0:
				var first_response = response_data[0]
				if first_response.has("generated_text"):
					var generated_text = first_response["generated_text"]
					print("Generated Text: ", generated_text)
					var inner_json = JSON.new()
					var inner_parse = inner_json.parse(generated_text)
					if inner_parse == OK:
						var inner_data = inner_json.get_data()
						if typeof(inner_data) == TYPE_ARRAY and inner_data.size() > 0 and inner_data[0].has("content"):
							var response_text = inner_data[0]["content"]
							print("Response Text: ", response_text)
							if callback_function and callback_function.is_valid():
								callback_function.call(response_text)
						else:
							print("Error: Unable to parse the content from the generated text.")
					else:
						print("Error parsing the generated_text: ", inner_json.get_error_message())
				else:
					print("Error: No 'generated_text' key found in API response.")
			else:
				print("Error: API response is not an array or is empty.")
		else:
			print("JSON parsing error: ", json_parser.get_error_message(), " at line ", json_parser.get_error_line())
	else:
		print("Error: HTTP Response code ", response_code)

func _on_back_to_menu_button_down() -> void:
	pass

And for the Notepad where the texts are displayed:

extends Sprite2D

@onready var input_box = $Input as TextEdit
@onready var output_box = $Output as TextEdit
@onready var button = $Submit as Button

func _ready():
	button.pressed.connect(_on_button_pressed)

func _on_button_pressed():
	var user_input = input_box.text.strip_edges(true, true)
	if user_input == "":
		input_box.text = "Please type something!"
		return
	var node2d = get_parent()
	if node2d != null:
		node2d.chat_request(user_input, Callable(self, "_on_API_response"), "random_scenario")
	else:
		print("Parent Node (Node2D) not found!")

func _on_API_response(response_text: String):
	var bot_response = response_text.replace(input_box.text, "").strip_edges(true, true)
	output_box.text += bot_response + "\n"

And i will upload a photo of the scene`s tree if there is a mistake there…somehow. Also where the random_scenario is chosen it doesnt print the scenario but “random_scenario”.


I highly recommend using a tool such as curl or Postman (if you want a GUI) to call the API directly and see what actual response you receive. Based on that you can determine if the problem is the data you gave the API to begin with, or if it’s something on their end.

Thank you! Will try that in a sec. :]

Well everything seems to be fine with the API. I got recomended a “Less powerfull API for that task” whatever that means…

There are many different LLMs out there, many of them are specialized for specific things, others will run much faster, some can even be ran locally without the need to connect to a specific service. For a small project like this, you might be able to get away with using a smaller model with less context that can not only run faster, but hosting it will likely be much cheaper too.

However, if the API worked fine in curl or postman, perhaps there’s something you’re doing wrong when handling the response in code? I would not recommend relying on chatgpt too much, instead, try to look at examples on how to use the HTTPRequest node, and start with a very simple request, and get it’s response.

1 Like