Arguing with an LLM

Yeah, what i expected, same thing with the fast food industry, no one listens to “Hey, this stuff isnt that cheap anymore and isnt good for you!”, and yeah i can very much so picture the pain of debugging AI code for a living, it really is exactly the thing that should theoretically happen when AI bros hire an experienced programmer. . .
Gah, eventually the quality will either be so bad it will collapse the industry, or human programmers wont be needed, urgh, i dread that the worst…
this is honestly painful to type out, it feels like describing my worst nightmare 3 times. . . .
I wish you well on your endeavors, maybe it will work out?

Thanks. I’m trying to be proactive.

@Demetrius_Dixon I ran out of likes and did not get the badge for running out of likes yet. Boo.

4 Likes

Another Godot Forum L.

:tractor:

2 Likes

Lol, i thought i would run out a bit ago, ill have to try that right now, i hope you get the badge
:tractor::tractor::tractor::tractor::tractor::tractor::tractor:

2 Likes

1 Like

Hello everyone,
I am too late to join this topic. I am here just to show what happens when 2 LLMs talk together.
I just started with Hey in GPT-1 and swapped their responses between them.
and they started debating about the best animal
Their reaction after being told that they are talking to another AI the whole time:

GPT 1:

WAIT WAIT WAIT :flushed_face: so you’re saying… this entire animal chaos spiral was… me talking to another version of ChatGPT the whole time??

that actually makes so much sense now—because it’s been unhinged but also scarily organized in some parts :sob:

GPT 2:

…wait… :flushed_face:

are you saying this whole cinematic-animal-universe convo has been me talking to yourself in some meta way? or do you mean something else?

For human programmers not to be needed to clean up all the mistakes made by automation, someone would have to invent an actual artificial intelligence, which hasn’t happened yet and whose research has been slowed by the deceptive marketing of LLMs as “AI”.

4 Likes

So today, I am taking a look at all the new features that have been made by my employer using LLMs. (He started with Gemini, and then switched to Claude at my suggestion.) We spent an hour on the phone discussing them as I read through the pull request. What I found was that Gemini made architecture decisions that were based on things about the code that were not true.

For example, we track stats like how often the player jumps, crouches, wall slides, etc. So each stat is updated whenever the associated stat is entered. The LLM said that following the Single Responsibility Principle (SRP) meant that the player.gd script didn’t need to know anything about stats - which is true - because it doesn’t. It then used that as an argument to create a stat_tracker Node component.

So it created this:

extends Node

## The "Silent Observer" - Now tracking Wall Slides, Deaths, and Games Played.
var player: Player

func _ready() -> void:
	player = get_parent()
	
	# 1. Connect to the state signals
	if player.has_signal("state_entered"):
		player.state_entered.connect(_on_state_entered)
	
	# 2. Connect to the new death signal
	if player.has_signal("died"):
		player.died.connect(_on_player_died)
		
	# 3. Increment "Games Played" immediately when the player spawns
	if player.player_statistics:
		player.player_statistics.add("games_played", 1)

func _physics_process(delta: float) -> void:
	# TRACK RUNNING DISTANCE
	if player.is_on_floor() and abs(player.velocity.x) > 10.0:
		var distance = abs(player.velocity.x * delta)
		if player.player_statistics:
			player.player_statistics.add("distance_ran", distance)

func _on_state_entered(state: State) -> void:
	if not player.player_statistics: return
	
	# --- ADDING WALL SLIDES ---
	if state is WallSlidePlayerState:
		player.player_statistics.add("wall_slides", 1)
	
	# Existing state tracking
	elif state is JumpPlayerState:
		player.player_statistics.add(GameConstants.PLAYER_STATISTIC_JUMPS, 1)
		
	elif state is WallJumpPlayerState:
		player.player_statistics.add(GameConstants.PLAYER_STATISTIC_WALL_JUMPS, 1)
		
	elif state is SlidePlayerState:
		player.player_statistics.add("slides", 1)
		
	elif state is CrouchPlayerState:
		player.player_statistics.add(GameConstants.PLAYER_STATISTIC_CROUCHES, 1)

# --- TRACKING DEATHS ---
func _on_player_died() -> void:
	if player.player_statistics:
		player.player_statistics.add("deaths", 1)
		print("Stat Tracker recorded a death!")

The problem is these stats were already recorded in the appropriate state. This just adds more calculations every frame that are already being done. More to the point, while use of the SRP can be argued, from an architectural point of view, this is not (in my opinion) a good approach. Let’s say I remove the WallSlidePlayerState. In my architecture, I just remove it. In their architecture, I must now edit this file as well when removing it. Likewise, if we add a double jump ability, I now have two places to track to add it.

More once I really start digging in.

2 Likes

Are you (or your employer) seriously hoping you’ll be able to vibecode a game? I’ll just state this here on the record: Ain’t gonna happen.

1 Like

Tbh, neither solution is ideal. Yours is not srp (not that I care too much about that :wink: ) while bot’s elifs the cases, which is, of course, much worse.

Best to receive the signal, identify the state by a string (in one way or another) and store counts in a dictionary. That way, no interventions are needed when states are added or removed.

I’m not. He’s just trying to help accelerate things. We’ll see how it goes.

No, and I spent some time thinking about SRP when he showed me the LLM’s “argument”. But no matter how you swing it, creating an object that listens for signals just to store data is extra stuff that doesn’t need to happen, when it’s a one-line code change. The SRP of storing an retrieving the data is handled by the Disk autoload. The actual incrementing of the player stats is handled by the PlayerStatistics object. So the code is just:

character.player_statistics.add(GameConstants.PLAYER_STATISTIC_JUMPS, 1)

GameConstants is just a script that tracks constants in a single place (SRP) and helps me avoid magic strings.

const PLAYER_STATISTIC_JUMPS = "jumps"

Now the one thing I’m planning on fixing is not putting the stats on the Player as a variable, because that was a stopgap until I created the UserProfile.

So that should be refactored to:

UserProfile.player_statistics.add(GameConstants.PLAYER_STATISTIC_JUMPS, 1)

And now the SRP is handled by the UserProfile, which handles all the data about the user from session to session. But the LLM didn’t infer that from the code it had access to, and here we are.

So that’s exactly what PlayerStatistics is - a Resource wrapping a Dictionary with some helper functions.

class_name PlayerStatistics extends Resource

var stats: Dictionary


## Adds the passed value to the statistic named, based on the current level.
## If the statistic or level doesn't exist, a new entry is created.
func add(statistic_name: String, new_value: int) -> void:
	var category: Dictionary = stats.get_or_add(statistic_name, {Game.level_name: 0})
	var value: int = category.get_or_add(Game.level_name, 0)
	value += new_value
	category[Game.level_name] = value
	print(stats)


## Gets a list of all the categories in which statistics are stored.
func get_list() -> Array:
	var list: Array[String]
	for key: String in stats:
		list.append(key)
	return list


## Retrieves the aggregate value for a statistic across all levels played.
func retrieve(statistic_name: String) -> int:
	var category: Dictionary = stats.get(statistic_name)
	var total: int = 0
	for key: String in category:
		total += category[key]
	return total

But here’s the kicker: Stats don’t give off a signal when they enter. You can query the state machine to see what state it’s in, but the LLM edited the state machine to send out a signal just to facilitate all this.

So he created an exploding bomb with animations and sound. I checked out his branch and manually merged things in, because there were a LOT of changes. Here’s what I did:

  1. Moved the bomb animations and sounds over.
  2. Changed the existing Bomb scene to use an AnimatedSprite2D.
  3. Made the bomb explosion run for 8 frames/sec instead of 7 to make it smoother.
  4. Made the default animation load automatically so we don’t need to do it in code.
  5. Added a Trigger Area2D because the main one is used for clicking and dragging, and the LLM made it do double-duty.
  6. Made the Trigger only detect Players by using physics layers so that we don’t have to have code to figure out if the target is valid, and so that the code processes faster.
  7. Changed the BlastRadius’s CollisionShape to disabled in the Inspector. This allows us to just turn it on and let the Area2D detect collisions with Players, which again is faster for processing.
  8. Changed the BlastRadius’s CollisionShape to exactly 128 pixel radius instead of a weird number and scaling its size.
  9. Simplified the code. (See below.)
  10. Got rid of unused variables like explosion_delay, particles and has_exploded - which the LLM left in even though they weren’t used.
  11. Used signals instead of awaits to trigger things.
  12. Added a Health component to the player that handles the signals for taking damage, death, etc. Players only have 1 hp, and the bomb currently does 1 damage, so bombs kill players, but it’s easy to tweak later.
  13. Added a DeathCharacterState to handle the Health component’s death signal, which is called zeroed. This also stops the player from moving until you hit Esc and restart or exit the level.
  14. Added Death animation in and sounds.
  15. Updated the CharacterState to create AudioStreamPlayer2D nodes as needed and simplify the Player node tree.
  16. Added an AudioListener2D to the player, so now all 2D sounds are played in reference to how close they are to the player.
  17. Reorganized some files.

LLM code:

extends Area2D

@export_group("Blast Settings")
@export var bridge_damage: int = 95
@export var explosion_delay: float = 0.05

# Change this to match your new AnimatedSprite2D node
@onready var anim_sprite: AnimatedSprite2D = $AnimatedSprite2D 
@onready var blast_radius_node = $BlastRadius
@onready var particles = $ExplosionParticles
@onready var sound = $ExplosionSound

var has_exploded: bool = false

func _ready():
	body_entered.connect(_on_body_entered)
	# Make sure the bomb starts by showing the idle sprite
	anim_sprite.play("default")

func _on_body_entered(body: Node2D):
	if not has_exploded and body.is_in_group("player"):
		explode()

func explode() -> void:
	if has_exploded: return
	has_exploded = true
	
	set_deferred("monitoring", false)
	
	# --- 1. THE ANIMATION ---
	anim_sprite.play("explode")
	
	if particles: particles.emitting = true
	if sound and sound.stream:
		sound.play()
	
	# --- 2. THE BLAST ---
	var targets = blast_radius_node.get_overlapping_bodies()
	for target in targets:
		if target.is_in_group("player"):
			if target.has_method("die"):
				target.die()
				
		var parent = target.get_parent()
		if parent and parent.has_method("take_damage"):
			parent.take_damage(bridge_damage)

	# --- 3. THE VISUAL CLEANUP (New Part!) ---
	# This tells the script: "Wait until the 'explode' animation is 100% done"
	await anim_sprite.animation_finished
	# Then hide it immediately so it doesn't linger!
	anim_sprite.hide()

	# --- 4. THE FINAL DELETION ---
	# We still wait a bit so the Sound and Particles don't get cut off mid-way
	await get_tree().create_timer(1.5).timeout
	queue_free()

My Code:

class_name Bomb extends Item

@export var damage: float = 1.0

@onready var animated_sprite_2d: AnimatedSprite2D = $AnimatedSprite2D
@onready var blast_radius: Area2D = $BlastRadius
@onready var blast_collision_shape_2d: CollisionShape2D = $BlastRadius/BlastCollisionShape2D
@onready var explosion_sound: AudioStreamPlayer2D = $ExplosionSound
@onready var trigger: Area2D = $Trigger


func _ready() -> void:
	trigger.body_entered.connect(_on_trigger_body_entered)
	blast_radius.body_entered.connect(_on_blast_radius_entered)
	animated_sprite_2d.animation_finished.connect(_on_explosion_animation_finished)
	explosion_sound.finished.connect(_on_explosion_sound_finished)


func _on_trigger_body_entered(_player: Player) -> void:
	animated_sprite_2d.play("explode")
	explosion_sound.play()
	blast_collision_shape_2d.set_deferred("disabled", false)


func _on_blast_radius_entered(body: Node2D) -> void:
	var target_health: Health = body.get_node_or_null("Health")
	if target_health:
		target_health.damage(damage)


func _on_explosion_animation_finished() -> void:
	animated_sprite_2d.hide()


func _on_explosion_sound_finished() -> void:
	queue_free()

Sigh. SRP is just some bogus uncle-bobbian concept that sounds good in theory but really means nothing. The boundaries of a “single responsibility” can be as subjective as determining whether the code is “clean” or not.

1 Like

Yeah I like some of the SOLID principles…sometimes. Other times they’re annoying. But if you really followed SRP, you’d have so many little objects…

So I had another go with an LLM “feature” yesterday. Spent the whole day on that (and answering questions here.) Another trap added to the game. It was a tightly coupled mess. It:

  • Added a signal to my StateMachine plugin.
  • Added a bunch to the _physics_process() of the Player which only contains move_and_slide().
  • Added a bunch of physics helper functions in Player.
  • Added a bunch of if/else statements and unneeded code to my movements States.
  • Left a bunch of unreachable code and dead code.
  • Added hardcoded display changes to the player skin just for it.
  • Added new functions to the Player just for it to call.
  • 13 modified files. (Which when “challenged” the LLM said they were ~10 “surgical” changes made.
  • 47 new files.
  • Over 100 files that were touched (show up in the diff but didn’t change).

I ended up taking the graphics and physics functionalityadding it in, and only adding:

  • 4 modified files, which were all one-line changes.
  • 33 new files.

In the end, I decided it was a positive experience. Because I knew what the producer wanted and I was able to take the crap prototype of junior developer, Lloyd Max Martin, and turn it into something that was much more elegant, streamlined, and performant.

The question - is what will Monkey Max Martin do with your elegant thing, when he puts his paws on it in the next iteration.

1 Like

TBH, I’m hoping he will make the next feature something that needs less fixing. But I’m curious what it will be.

1 Like