Streaming system that doesn't use nodes, only the Servers

[byteseb]

Godot Version

4.7.1

Question

Hey. I’m working on a survival sandbox game with procedural small, but dense cities.
I wanted to have a destructible map and base building mechanics. So something like Fortnite, Minecraft or Valheim. There is no terrain deformation, and the player builds by placing modular pieces (the buildings are also made out of them). I was hoping some of you could guide me in the right direction.

The problems

I relied heavily on nodes. But that meant every wall had nodes for its mesh, static body, collision shape, hurtbox and occluder. Of course, this lead to an explosion of Nodes (usually, 100K+) when actual cities where generated. Most were Meshes, static bodies and collision shapes with no scripts.

I was also reaching the Static Body limit easily. I can increase it in the project settings, but it felt like a workaround.

My solution

So what I came up with is a Chunks system that doesn’t load nodes/scenes and that is inspired by an ECS architecture (I know it’s not true ECS).

Each Chunk can contain many entities. Entities are serializable Resources that can contain Components, that are only data containers, and these are processed by Systems.

For example, a bricks wall would be an Entity containing a MeshComponent, a CollisionShapeComponent, DestructibleComponent and OccluderComponent. When the player builds a wall, the systems are notified and call the RenderingServer and PhysicsServer to process that new entity.

When a Chunk is reloaded, it does the same thing, but for all its entities.
This architecture is mostly event based, so the game is not iterating every frame. If a mesh moved, the MeshSystem only updates that mesh once, when it actually changed.
Instead of having a single static body per wall, now I use a single static body per chunk and submit the collision shapes of each entity as compound shapes.

It still needs a bit of polish. It can stutter when streaming a large number of entities at once. This could be solved by loading them across frames or perhaps using threads.

Cons

1. Handcrafted content is clunky to make. If I wanted to design a handcrafted room, I can do it, but the best solution I could find is to design a scene as a “template” and run a tool script that bakes it into Entities with the corresponding components.

2. I lose the functionality of scenes and nodes. Most of the core features are part of the Servers, but things like AnimationPlayers, IK, etc are exclusive to nodes. I try to keep compatibility with Nodes by using a SceneComponent. It’s for the minority of cases that actually need it.

3. I lose debug information. The debug meshes for Collision Shapes and Navigation Meshes is exclusive to nodes too. So I just can’t see if something is working as intended that easy.

4. I have to duplicate some code. MeshComponents, LightComponents, OccluderComponents, MultiMeshComponents, etc. All of these are basically a data driven equivalent to MeshInstance3D, Light3D, OccluderInstance3D and MultiMeshInstance3D. Same properties, just following a different philosophy. Which is why I feel like part of it is reinventing the wheel.

Another thing to keep in mind is that even though, all pieces of the same type are rendered in a single draw call thanks to Godot’s GPU Instancing, I submit them to the RenderingServer as MeshInstance3Ds, not as a single MultiMeshInstance. Would this be an issue? I do it this way because it’s more flexible and also allows for shader instance uniforms, LODs and frustum culling of individual instances.

What do you think? Is there a better solution? Am I over-engineering all of this? I love the Scene tree, but for something like this, using nodes might not be the best solution.

[stone]

It sounds like you want to make a LOD system.

The screenshot you presented of the ECS system makes it look like you’re reimplementing the way a scene stores things. Personally, I would rather use a scene, then create intermediate representations from that scene. From that intermediate representation you can have LOD0 - the same way the building system would work, LOD1 colliders and meshes simplified, LOD2 basically a cube with no collisions etc. etc.

In other words, the intermediate representation can be used to then calculate various LODs with different levels of simplifications (both of rendering and collisions). In addition, doing it this way allows you to reuse the same logic the player would use to build the base - as eventually it would be made into a LOD chunk.

As you are dealing with voxels, you can use voxel-based optimisation methods to simplify the meshes/collision shapes. You can summarise the collision shape more than one voxel as a cube. If e.g. a projectile from far away hits it, you know that it will hit the sides/top/bottom and can do some calculations (worst case, instantiate a LOD0 and figure out from there).

LOD0, LOD1, LOD2 would just be standardised nodes with a fixed number of mesh renderers, collision shapes etc. They would be responsible for taking the intermediate representation into an actual chunk.

I suggest coming up with representative test cases you can test your solution against, especially performance wise.

I do think you are overthinking the rendering. I wouldn’t think a bunch of textured quads would take up that much performance. From the screenshots you presented, the total number of nodes may not be the best metric to optimise against - especially as you’re at a very respectable frame rate. Though I don’t know enough about this aspect of Godot.

About the number of the static bodies, I don’t know how the default limit was determined. If you’re using Jolt physics, have a look at their docs (for Jolt itself first, before seeing if there are any limitations on the Godot specific implementation).