Load PCK, DirAccess only uses new PCK directories?

Godot Version

Godot 4.2.1 - Linux

Question

Alright. This is a bit obnoxious. I have a system in which I load a database using DirAccess for debugging purposes. Another thing I have is a patcher that applies assets from a separate PCK, intended to override assets in the base game, but not scripts or logic. The reasons are irrelevant but I am configuring the project to let us release a minimal patch to the game in the future that has alternate graphics.

There’s an annoying catch I’m seeing. I have a toggle for applying this asset pck patch on game boot (both in editor and on build.) If it’s false, the PCK won’t be patched on. In this state, DirAccess can retrieve assets from a database. For example, res://db/mons/ for a monster database. This just contains a collection of custom monster resources. After giving DirAccess the path, everything works fine.

If, however, the PCK patch is loaded, then DirAccess can no longer access res://db/mons/. This happens regardless of how the PCK is loaded (whether replace_files is true or false.) The expected behavior feels like it should be merging the directory lists, but it feels as if it only uses the patch’s PCK directory, which seems bewildering of a choice and an oversight. So I did a test and had DirAccess print its contents both before and after loading the PCK, and indeed, DirAccess only prints the PCK’s image overrides with res:// content, entirely forgetting what the root PCK and project files are.

Is there a way to have the base DirAccess preserve regular res:// paths while also loading minimal PCK overrides? This feels like an issue when it comes to using pck file patches and overrides. I’m under the impression I’d even call it a bug with how unexpected it was.

If you don’t show how your patcher works, this is very hard to answer.

func _apply_patch_file(patchfile: String) -> void:
	if not DragonRPGDataBlock.instance.enable_patch_pck:
		return
	if not FileAccess.file_exists(default_patch_file):
		return
	var success = \
		ProjectSettings.load_resource_pack(default_patch_file)
	if not success:
		printerr("Error %s loading patchfile %s", [success, patchfile])

Not much to really show besides the standard load_resource_pack. enable_patch_pck is the option that toggles it for convenience’s sake. This is the first thing to run in the game’s boot scene, before the game’s kernel and everything else is initialized.

For some extra context, here’s what the resource loader looks like:

class_name G_DATABASE
extends Script


## This is more of a dev feature. We don't care about
## allocation slowness or caching.
static func get_resources_from_database(db_path: String) -> Array[Resource]:
	var res: Array[Resource] = []
	#var dir: DirAccess = DirAccess.open(db_path)
	#print(dir.get_files())
	## BUG We can't access anything this way if we load the patch pck.
	var files_raw: PackedStringArray = \
		DirAccess.get_files_at(db_path)
	for fname in files_raw:
		var new_entry: Resource = load(db_path + fname)
		if is_instance_valid(new_entry):
			res.append(new_entry)
	return res

Can ResoueceLoader still load resources from root PCK?
Did you try DirAccess.open?
Try to use a index file?

Besides the patchfile argument not being used in _apply_patch_file(), you’re using default_patch_file, I am not sure what is wrong here. If you’re passing a string starting with “res://” as a db_path it should give the same result after patching.
It could be a mistake with the path or some edge case with overriding directories.

What I would do is try to make a new project to reproduce this with minimal data so you can pinpoint if it’s a bug or a mistake or some other weirdness.

DirAccess.open() only works for one file. We need the directory to iterate through each file inside to dynamically fetch the database entries.

Root PCK can’t really be accessed in-editor, so that’s a no-go (unless there’s something I’m unaware of)

Index file, I’d probably have to look into.

I think I’m just gonna have to do that to present my case, yeah. Of course it’ll probably find some way to make me look dumb and magically work, but I’ll make a minimal demo real fast.

func dir_contents(path):
	var dir = DirAccess.open(path)
	if dir:
		dir.list_dir_begin()
		var file_name = dir.get_next()
		while file_name != "":
			if dir.current_is_dir():
				print("Found directory: " + file_name)
			else:
				print("Found file: " + file_name)
			file_name = dir.get_next()
	else:
		print("An error occurred when trying to access the path.")

Did you try this?

(edit)I found it in document:

Note: When used on a res:// path in an exported project, only the files actually included in the PCK at the given folder level are returned. In practice, this means that since imported resources are stored in a top-level .godot/ folder, only paths to *.gd and *.import files are returned (plus a few files such as project.godot or project.binary and the project icon). In an exported project, the list of returned files will also vary depending on whether ProjectSettings.editor/export/convert_text_resources_to_binary is true .

(edit)If ProjectSettings.load_resource_pack changed root PCK file or its reference and work of DirAccess depends it, it will cause that problem.

This is what I was concerned of. How obnoxious. I don’t think anything can be done for this without a specific parameter for load_resource_pack to merge and not entirely destroy the original file layout. I don’t know why this is the default way it works considering it seamlessly applies file overrides perfectly fine without destroying literally the entire game. But I also haven’t much idea of what it looks like on the C++ level.

You should file an issue in the bug tracker. This is likely to get a fix either as a behaviour change or a parameter added. Seems important to be able to do incremental patching easily.

From asking on the engine dev chat, I got this and a related attempt to fix this about two weeks ago, which seems inactive or the discussion happening elsewhere. I don’t think there’s anything that can be done for now.

Two weeks is not a long time for a bugfix. They will get to it eventually. If you know C++ maybe you can help. I do not know enough.