No because the null hypothesis is that it does work, you can’t just throw out claims or concerns that aren’t reasonable without pointing to some reasons to believe it
But you seem intent on discrediting the engine and making claims about it not being capable so I’ll leave you to it
I’ve had problems with large files and scenes over 1–2 GB. So I do have reason to doubt that Godot can handle large files.
So you don’t have any real examples? That makes everything clear.
If a single scene file of yours is a gigabyte then it’s certainly setup wrong, you may have accidentally built-in a expensive resource like images.
“Road to Vostok” lists >10GB on it’s steam page, I don’t see any issues with the game. If you know of a released game in Godot that is just falling apart because of it’s large pack file or a blog post lamenting Godot’s single pack file architecture, then you have some ground for an argument but as it stands this whole thread is strange speculation and no basis. If you’ve worked with system calls you know one big file is one big advantage; every game engine since DOOM makes a pack file.
It is also completely irrelevant to the topic of the limitations of pck files, and if you have that large scene files then having single or multiple pck files doesn’t matter either
Disk fragmentation is just the filesystem deciding to split your big file into several smaller files for you. It’s not ideal for performance, but it’s still better than splitting the big file into smaller files yourself, because you’ll end up with fewer fragments.
Having read this discussion I think @Master172 asks a decent enough question concerning their growing project size. The concern was real for them. However I think @athousandships answered you perfectly.
So this seems to be very much a ‘cross that bridge if you come to it’ type situation given that not only may you never find that bridge, but that ‘the bridge’ probably does not even exist.
While there might be value for reasons of organization and potentially performance in having multiple separate pck files for different parts, it isn’t possible to unload pck files currently, so it wouldn’t really work to switch areas of a game or dynamically, so there wouldn’t be any real benefit to having separate pck files to handle some upper limit etc.
Having your assets split into multiple pck files is almost certainly going to be slower than one large file as there’s an overhead for each file to look up. Any disk i/o issues should be resolved by having robust loading processes anyway, so you shouldn’t have the load time of individual files be a bottleneck in your project, regardless of the size, to accommodate slower hardware (i.e. loading on a thread, preloading resources, etc.), but there shouldn’t be any significant differences loading individual files in a very large pck file unless the underlying loading is slow, but that won’t really make a difference based on the size of the pck file
Seeking in a pck file should be pretty efficient and they are split up into sections, so it isn’t like looking at a single giant binary blob, as it has its own virtual file system solution, the size of a pck file shouldn’t have any significant effect on memory usage either, though the size of directories in it might affect things but probably only marginally
tl;dr; as far as I can tell looking at the implementation there’s nothing inherently causing issues with very large pck files with respect to the ability of Godot to handle it, or performance issues
I think you misunderstand, a single .tscn scene file over >1GB is out of the ordinary. Using 8K textures and lidar-scan models to compose a (in total) >1GB scene is not. If you are having trouble with the latter you can post in Help with some examples, but in general such a high quality game will be hard to run.
This is still not strictly relevant to large pack sizes, you could have a >100GB pack file with tons of small levels of disparate content. One large scene of 5GB high quality assets is more of a VRAM issue than disk i/o.
This isn’t really a game scene, but rather a demo showcasing both the engine’s capabilities and the underlying technologies. I don’t plan to include a scene like this in the game. But as a test, it works just fine. I’m going to streamline the scenes as much as possible, so I might not push things to the limit (if there even is one).
Yeah, that sounds like how I’d use it.
@Master172 Best of luck in your game projects! I’ll leave you all to it and I hope what I explained helps you get a better understanding of the potential issues and benefits of different approaches and what you might expect
And hoping you’ll share any potential future projects for the community to see, I’d love to see what you end up making! It is why we make this engine after all!
I don’t know about general .pck question that you asked, it but it is definitely sure that if you try to reduce and some unwanted (textures which you don’t see generally) texture sizes and making normal optimizations on game assets would also reduce your concerns regarding the .pck, this will help indirectly, and of course make your game better
I’m not sure if someone already mentioned it but exported file size may be the least of your concerns with these sizes. How are you going to load this in memory? If you only have a handful of models with this size I assume these are huge models with huge textures. If you try loading too many of these at once you will choke your computer’s RAM and VRAM way before you hit any export size limits, so you may want to look into some other areas for optimization.
Having said that, I see Road to Vostok mentioned here. There many other examples to check here: Showcase – Godot Engine (they don’t list size, you’d need to check one by one). Another commercial use case on the larger size is Sonic Colors which I believe is around 10gb. They even managed to do this with Godot 3. They used a fork, but AFAIK, looks like they relied on Godot for the file management as well. You can read about it here:
Load what? The size of individual assets or very many loaded at once is not really the concern raised here but the size of the pck file
The situation is as follows: there are several regions that are not directly connected to one another, each consisting of loadable areas and having little in common with one another. (For example: the Sahara and the Arctic.) And there is a group of travelers. To move between regions, the group must reach a specific point (let’s say, an airfield), and then they can travel to another region. The group changes their clothing and gear. These regions and gear can be included in DLCs and released as add-ons over time. How this works in the engine is clear—one scene is unloaded and another is loaded. But here’s the question: should we aim to pack all the content into a single .pck file, or is it better to split everything into parts? When re-releasing the game, should the finished DLCs be included in a single .pck, or is it better to keep them separate?
DLCs could fall under separate Pack files, if that’s how you choose to distribute DLC. It’s a pretty common approach to pack all DLC in the base game and only unlock it with valid entitlement.
Separate packs are an option, they are going to be harder to manage than one pack. Personally I’d only use it for loading community content such as mods.
Note that until https://github.com/godotengine/godot/pull/61286 gets merged using separate PCKs is not very effective as you can’t unload them once loaded
I don’t understand why unloading pck files would be necessary or even useful for anything other than disabling mods. Loading a pck file should use next to no memory because it only loads the index of the pack, not the actual resources in the pack. In the best cases, unloading a pck file does nothing other than shuffle around indexes because none of its resources are in use. In the worst case, it breaks the game by purging resources that are currently in use.
If you use multiple PCK files to handle large data then what’s the point of not just using one big file if you can’t unload them when done? For example when moving between chapters, see the comment above “The situation is as follows:”
To be clear I know PCK files don’t take a lot of memory, hence why splitting files isn’t necessary
I agree that splitting a game into multiple pck files usually isn’t useful. However, there are exceptions, such as:
My point is that for those exception, there is no need to unload pck files.
Agreed but that wasn’t the point I was responding to