~500,000 Word English Dictionary(~5MB) Fails To Load On Web HTML5 ? Runs 100% On Desktop - "LettersFall 110%™" - MIT

[savantsavior]

Godot Version

v4.5.1.stable.official [f62fdbde1] Arch based CachyOS Linux distro x86_64

Question

Hi,

Working now on Beta 1 of “LettersFall 110%” educational word spelling game using Godot v4.5.1.
The below source code fails to finish on Web HTML5, but on Linux x86_64 it finishes in about 2 seconds?

How would I fix the loading of the American English word dictionary on Web HTML5 version?

WordDictionary.resize(500000)

var endOfFile = false
var file = FileAccess.open(“res://media/Dictionary/words.txt”, FileAccess.READ)

var index = 0
while (endOfFile == false):
    WordDictionary[index] = file.get_line()

    if (file.eof_reached() == true):
        endOfFile = true
    else:
        EndOfWordsFile+=1
        index+=1`

Any help would be appreciated, very close to Beta 1…

[gertkeno]

Can you tell where it fails? Do you get any errors?

[savantsavior]

It takes to long to load on Web HTML5.
After about 10 seconds I get a browser warning that the page is unresponsive?

It’s a ~5MB single TXT file English word dictionary (~500,000 words).

The code above is done once at start of Web application.

NOTE: The Web HTML5 game in it’s broken state is live on Itch below
https://teamjezxlee.itch.io/lettersfall

[gertkeno]

You can press F12 in the browser to open the developer console, it seems to load the file okay so I’d guess somehow you have an infinite loop. You could see print statement appear in this console if you add them into the code.

[savantsavior]

Hi,

Thanks for helping.
On x86_64 Linux desktop the dictionary loads properly?

Perhaps I should not load ~500,000 lines of a TXT file in a loop?
Maybe release control to browser periodically?

How would I do the above using a single words.txt file?

[gertkeno]

In my own tests I can load in 370,107 words on web about as fast as it loads on desktop. 500,000 words really isn’t that much to a computer, and you’ve already optimized a by resizing the array before use.

This is the code I used.

func loadWords() -> void:
	var file = FileAccess.open("res://a_combined_list.txt", FileAccess.READ)
	assert(file != null, "Failed to open word list")

	words_array.resize(500_000)

	var idx := 0
	while not file.eof_reached():
		words_array[idx] = file.get_line()
		idx += 1
	var success := words_array.resize(idx+1)
	assert(success == OK, "Failed to resize by trim")

Add some print statements, show more of your code.

[savantsavior]

Tried above code, no difference.
I added a print(idx) and it displays slow increments in browser console?

It is not an endless loop on Web HTML5, just the half a million get.line() is too slow.
I am on a mini PC with CachyOS Linux distro OS.

[gertkeno]

Are you running out of RAM? It could be writing to swap which would be significantly slower.

[savantsavior]

RAM memory is stable, swap never activated…

Not sure what to do here, please advise.

Project GDScript source code is uploaded to GitHub below:
https://github.com/savantsavior/lettersfall?tab=readme-ov-file

NOTE #1: Game ONLY runs properly from the engine IDE editor,
it fails on Web HTML5 AND Linux x86_64 desktop?

[gertkeno]

For me, this assert fails.

var file = FileAccess.open("res://media/Dictionary/words.txt", FileAccess.READ)
assert(file != null, "Failed to open word list")

TXT files are not exported by default, you must add them to the exports list. Afterwards the game runs fine, though I’m not sure how to play.

Make sure to avoid uploading your .godot folder to git, it is very large, constantly changes, and can contain paths in your personal filesystem.

[savantsavior]

That was the issue! Many thanks….