Best Practises When Using HTTPClient To Get Data?

Help Networking
, ,
1

Godot Version

4.4.1

Question

I am updating my music app to be able to request the latest public version from the Github repo so the user can see if there has been any updates, I have managed to get it working but as this is my first real dive into HTTP/Internet things in programming I was wondering if there are any good practises / must-knows when it comes to handling errors / exceptions?

Currently this is my code, I have tried to handle errors by reporting all as failed opposed to trying to resolve it
I have the awaits to avoid checking the same things every frame unnecessarily:

func attempt_repo_connection() -> bool:
	## 'cli_print_callable' prints something to the CLI present in the app.
	cli_print_callable.call("SYS: Attempting to connect to Github API.")
	repo_http_client.connect_to_host("https://api.github.com")
	var continue_attempt : bool = true
	var attempt_start_time : int = int(Time.get_unix_time_from_system())
	
	while continue_attempt:
		await get_tree().create_timer(0.25).timeout
		repo_http_client.poll()
		await get_tree().create_timer(0.25).timeout
		
		match repo_http_client.get_status():
			HTTPClient.Status.STATUS_CONNECTED:
				repo_connection_succesfull = true
				cli_print_callable.call("SYS_ALERT: Succesfully connected to GitHub API.")
				return true
			HTTPClient.Status.STATUS_RESOLVING, HTTPClient.Status.STATUS_CONNECTING:
				## If it has taken over 5 seconds and we are still requesting, mark it as failed, otherwise continue
				continue_attempt = int(Time.get_unix_time_from_system()) - attempt_start_time < 5
			_:
				## Something wrong happened; stop trying
				continue_attempt = false
	
	cli_print_callable.call("SYS_ERROR: Unable to connect to Github API, if you would like to attempt to reconnect please run 'call-_attempt_repo_connection'.")
	repo_http_client.close()
	return false

func get_latest_app_version() -> void:
	if not repo_connection_succesfull:
		cli_print_callable.call("SYS_ERROR: Attempted to get latest app version without being connected to the repo, please try to reconnect to the repo using 'call-_attempt_repo_connecton'.")
		latest_version = "Unknown"
		return
	
	cli_print_callable.call("SYS: Attempting to fetch latest app version from Github.")
	repo_http_client.request(HTTPClient.METHOD_GET, "/repos/NatZombieGames/Nat-Music-Programme/releases", ["accept:application/vnd.github+json", "X-GitHub-Api-Version:2022-11-28", "per_page:1"], '{"owner":"natzombiegames","repo":"nat-music-programme"}')
	var continue_attempt : bool = true
	var attempt_start_time : int = int(Time.get_unix_time_from_system())
	
	while continue_attempt:
		await get_tree().create_timer(0.25).timeout
		repo_http_client.poll()
		await get_tree().create_timer(0.25).timeout
		match repo_http_client.get_status():
			HTTPClient.Status.STATUS_BODY:
				
				match repo_http_client.get_response_code():
					HTTPClient.ResponseCode.RESPONSE_OK:
						var response : Dictionary[String, Variant] = {}
						## Get the response chunk, turn into utf8 string, parse it as JSON before getting the first item.
						response.assign(JSON.parse_string(repo_http_client.read_response_body_chunk().get_string_from_utf8())[0])
						cli_print_callable.call("SYS_ALERT: Succesfully retreived latest app version from Github.")
						repo_http_client.close()
						repo_connection_succesfull = false
						## Each release name is the version, so we can just grab the name
						latest_version = response["name"]
						return
					_:
						cli_print_callable.call("SYS_ERROR: When fetching the latest app version received the response code '[u]" + str(repo_http_client.get_response_code()) + "[/u]' which is unhandled, unable to get latest app version.")
						latest_version = "Unknown"
						return
						
			HTTPClient.Status.STATUS_REQUESTING:
				## If it has taken over 5 seconds and we are still requesting, mark it as failed, otherwise continue
				continue_attempt = int(Time.get_unix_time_from_system()) - attempt_start_time < 5
			_:
				## Something wrong happened; stop trying
				continue_attempt = false
	
	cli_print_callable.call("SYS_ERROR: Unable to get latest app version, either received unhandled status code or it took too long to connect. Received status code '[u]" + str(repo_http_client.get_status()) + "'[/u].")
	latest_version = "Unknown"
	return

This seems sufficiently rigorous to me but I am not sure, so I was wanting to see if there are any important things that should be kept in mind when making simple requests like these?

Thanks

2

This is awesome to see. I’ve done a lot of web programming over the years, but nothing with Godot.

I would recommend writing unit tests for each of these scenarios using the built-in C++ unit testing or something like GDUnit4 (my personal preference). This is going to help you when you, or someone else has a problem connecting to get updates. First thing you do is run your unit tests. It will save you a ton of debugging time, especially useful if you’re panicking about getting things working again.

Writing unit tests will also help you identify places where you may have gone overboard. If you’re writing the same test twice in different ways, you may end up simplifying your code just so you don’t have to write more tests.

Specifically related to that, while I don’t see the relation between attempt_repo_connection() and get_latest_app_version() in your code, logic tells me you run the first one before running the second. Therefore this code appears to be redundant and should never return:

	if not repo_connection_succesfull:
		cli_print_callable.call("SYS_ERROR: Attempted to get latest app version without being connected to the repo, please try to reconnect to the repo using 'call-_attempt_repo_connecton'.")
		latest_version = "Unknown"
		return

Because if the connection fails, your code that is calling both of these should know that and never call the second function. This is a good example of the Single-Responsibility Principle in action. attempt_repo_connection() handles connecting. get_latest_app_version() handles what happens if you connect. Outside of that it’s another functions responsibility to utilize these functions and deal about their outputs. get_latest_app_version() doesn’t need to know anything about the concept of “connecting”.

About the awaits. I’m not sure why you have one on either side of your polling call. It’s that same as just putting await get_tree().create_timer(0.5).timeout Also, it is WAY too long a pause. Unless this is somehow bringing down the functionality of your program, I recommend you just take them out. I used to write tests that would make calls to web pages, get the page data, enter data and submit it, and get a response - all in under 200 milliseconds. Web calls are much faster than even most web developers realize. The things that slows them down is connection speed. Have you printed out the accumulated time it takes to connect and watched it? Converting it to milliseconds would be informative I think.

continue_attempt = int(Time.get_unix_time_from_system()) - attempt_start_time < 5

5 in this line of code is what’s known as a “magic number”. That 5 only makes sense because you have a comment. However if you were to add this to your code:

const TIMEOUT = 5
...
continue_attempt = int(Time.get_unix_time_from_system()) - attempt_start_time < TIMEOUT

Then your meaning becomes a lot more clear. Also, if you want to change the timeout (which I think is too short), it’s easy to do. 10 seconds is probably a better number because you don’t know how bad someone else’s internet connection is. (I once worked for a company that was owned by a German company. All our work internet traffic was routed to Germany and back before going out to the internet. Our lag was reliably long.)

Finally await stands for “asynchronous wait”. This means your program will keep functioning while it runs. Try calling:

await.attempt_repo_connection()

And removing the awaits from inside the function. Then when your timeout is reached, the code will handle a failure, but the rest of your program will continue doing whatever it’s doing (like loading, playing music, etc.)

1 Like
3

Thank you! You were very helpful and informative

The reason I had that is I allow the user to call these functions themselves, so I wanted this in the case that someone tried to call the latter before the former.

I never knew that it could be that fast, this whole time I had assumed web stuff wasn’t that fast unless you were really good at optimising it; I was able to remove the awaits and it worked just as well except now in a fraction of the time!

I was also able to implement what you mentioned about making the 5 seconds to reconnect 10 and changing it to a constant, and when it comes to unit tests I have never deeply looked into them so that will be something I investigate further in the future.

Thanks for the help!

4

Ok, sure I get that impulse. I also like to idiot-proof my code (especially for my future self). However, what I would do is change the second function to test for a connection, and if it isn’t there, run the first one. Making the connection for someone is much more helpful than throwing an error they’ve got to track down. (Especially future you.)

I once went to a company and turned an automated full site regression from a 2-hour ordeal to a 5-minute test that was more complete. A lot of people make those assumptions. As humans really hard to understand how fast computers can do things.

Adding unit tests will help with that because it will tell you how fast each test runs.

Awesome! Glad I was able to help! Good luck!