I had a question wondering how Godot handles large projects with a lot of data that needs to be contained in PCK files
i admit i have not reached at a point where i have to worry about the PCK file being so large that Godot cannot handle it properly
but still i wonder how would Godot handle exporting a game that has 10gb worth of resources to be exported (Images, Audio, Animations, Models, etc).
will Godot create a single PCK file no matter how large it gets, or will Godot start creating a GAME_NAME_01.pck and GAME_NAME_02.pck has the export pck size exceeds a certain limit, like unreal engine might create multiple PAK files.
or do we have to split the game into multiple PCK files ourselves?
i am currently working on a project for which i create assets myself (3d models, animations , textures, sounds, etc) , the project has not reached a point where this would be a problem, however i have seen its size increase quite easily whenever i create new assets (with just 5~6 weapons and 3 character models it was already 1.21 gb on disk) and i do plan to work on it as much as i can.
this might just be a what if question but i feel that knowing how Godot would handle this would not be a bad thing and just increase my understanding of the engine.
Got it, the framing of the question just seemed to expect that there would be problems and looking for ways to work around a problem that’s assumed, and was curious if you had heard claims of this somewhere
As far as I am aware there aren’t any issue with this, and splitting into separate pack files would be necessary
you can do it be going to the export tab and at first only exporting the code, .tscn files and .res files by adding all other file types to the ignore list
and then exporting all other file types in multiple pcks by everything in the export tab and only exporting files with certian file types as PCK’s
and then loading those pck’s through an autoload when the game starts in an empty scene and then switching to your main scene
The file size could cause problems — for example, when distributing the game. Should we archive it and split it into volumes? With a very large file, there could be errors during extraction.
It would be a bit disappointing to work on a project and then suddenly find out that it can’t be distributed.
This is one possible solution, but splitting the files by type will result in very uneven sizes for these PCK files. Although you can manually specify specific files, that involves a lot of manual work.
i did read on an a few old reddit posts that Godot cannot handle large pck files properly, though admittedly all those posts are 6 years old now, and the file limit on 2.1 gb for pck files should have been solved for 4.0
not only that but extremly large files are just not that easy for file systems to handle in general, due to stuff like disk fragmentation and high costs for indexing into large files
not to mention a lot of old file formats cannot even handle files past 4gb
in general i have some reason to believe that having your entire game be one large pck is undesirable, hence most games usually split their data into multiple files.
That is probably not for pck files but embedded pck files that are affected by Windows limitations for executables
Sure but they aren’t relevant to Godot really, you’re not going to be running Godot on ancient fat32 systems and similar I’d say
Are they really? They are usually in a single file or lots of files, plus additional files for expansions etc. I don’t know of any examples of games with several packed files like that for a single release
I think you’re trying to explore solutions for a non-existent problem or a problem that’s not going to be remotely likely, or not possible because of other limitations being more critical (like trying to run a Godot game on Windows 98)
I do agree bringing fat32 here was not a practical concern, however this still does not address wether Godot and the operating system can handle those large files properly due to the aforementioned issues with disk fragmentation (should only be a problem on HDD’s) and file indexing, parsing and searching (more of a RAM and CPU issue)
Splitting the pck into multiple files is also beneficial
I do not know how exactly pck files are passed but a larger pck file in general should mean more time spent on parsing and searching, hence slowing your games load times
Smaller and multiple pck files should help in load times for large games
And the biggest benifit with multiple PCK files is the fact that incremental updates and patches would take less bandwidth to download
I do know that godot can handle patches with split pck files but you will still have to edit the games source code to load those pck files hence updating the original pck,
Not to mention a pck cannot provide the simplest way to patch everything since some patches will require a change to the original pck, updates as well
I guess you could use steam here since I heard that steam only downloads the modified parts, but this also means smaller updates and patches are tied to a singular publishing platform
It can handle it safely, and multiple pck files won’t improve read times, you still have to locate the data regardless so splitting it up doesn’t make any difference, unless you split it up based on usage you’d have more processing going through multiple pck files at the same time, not less processing. This isn’t really going to be an issue on modern systems as far as I know
How would a single pck file take more ram or processing?
So I’d suggest focusing on the workflows and distribution and not worry about this hypothetical problem
Provide some evidence for that large games are not possible with Godot, instead of assuming they cannot be made, this is just unhelpful and not constructive to make unfounded claims of lack of support
The burden of proof lies with the person making the claim. If you say that “large games” work, then it shouldn’t be too hard for you to give some examples.
If you could provide examples of large games, the issue could be considered practically settled.