To fail and learn why the usual way works in a thorough way. I do it all the time. More fun.
4 Likes
Should we search for an experimental alternative to Pythagorean theorem every time we’re trying to calculate a hypotenuse of a right triangle?
Whats the best way of scattering trees and grass?
Define “best” and define “scattering”.
Picking optimal solutions require clear specifications.
Given the overall vagueness and dispersed focus of this whole discussion, I posted a good general approach here Handling Data Streaming & Chunk Loading for Large 3D Maps in Godot - #156 by normalized
1 Like
Are these problems of the same order of magnitude?
Actually, you’ve already answered that yourself.
Define “order of magnitude”.
How many ways are there to solve the Pythagorean theorem, and how many ways are there to implement asset dispersion and data streaming? Is there a solution for dispersion and data that’s just as simple and clear?
Ah ok … And how should they be drawn, instances with multimesh or static meshinstances that rely on Godots internal instancing?
Should the instances be entire Scenes or .res files?
1 Like
You can draw them however you want. That has nothing to do with scattering per se.
If you need suggestions for actual drawing optimizations, we’ll again need to go specific - visual style, asset variation etc. In vague and general terms - it’d be a no-brainer to multi-instance as much as you can and to keep the instance count as low as you can.
1 Like
Yes. I already said it’s straightforward and described how to optimally do it. There can be variations but the gist is pretty much the same. The important thing though is that solutions exist and there’s no need to reinvent the wheel except for educational purposes. In which case what’s the point of asking how to do it? Go, experiment and (re)discover.
Btw scattering some points around a heightfield is a much simpler problem than calculating the hypotenuse. You may think the latter is very easy because you’ve been given the recipe in the elementary school. To see just how hard it is - try to calculate it without using what Pythagoras and trigonometry dudes gave you.
2 Likes
Is the evolution of computer science comparable to mathematics?
Yes, certainly. There are standardized ways of handling memory management, physics, rendering pipelines, and optimization.
However, I would like to propose an alternative solution.
While it is obviously not the most efficient method, it works: for a given mesh, every single vertex/point receives an empty object in Blender.
This setup is then exported to Godot, where you can spawn objects interchangeably using these transform points.
What would be the downsides to this approach? Are there any potential upsides?
Assuming the vertices are arranged in a regular grid, how do you produce randomness? How do you eliminate unwanted positions (dependent on slopes and biomes)? You’d still nee to do something very similar to what I have described, only in Blender scripts. Unless you plan to place everything manually.
Ultimately, you only need a database of transforms and their corresponding assets, then stream that according to some space partitioning scheme. This database can entirely be pre-built, in which case the generation method doesn’t really matter, at least not performance-wise.
Throughout this thread you seem to be constantly mixing up generation and streaming. Those are two different problems. Generation is fun to play with but easy and not performance critical if you stream/tread it. Streaming is a much more challenging but “boring” problem.
You run in circles because you “research” too much. I’d suggest biting the bullet and implementing the simplest streaming system you can think of, that streams database of premade data, so that generation of that data is out of the picture. You can experiment with all the fun methods of generation after the streaming starts working.
Unless you want to go fully procedural route. In that case, you don’t really need any premade data streaming as everything is generated from code in real time. But again, the efficiency of generation is not really important. You’d still be “steaming” the generated data which gives you quite a bit of leeway to do it “slowly” from worker threads.
1 Like
The points originate from Geometry Nodes and are refined to filter for valid instantiation points. Saving them as a database makes sense, as the CPU will still need to handle them in either scenario.
My original focus was on the technical solution, which covers data generation and creation but primarily focuses on streaming.
Speaking of this ‘boring data’—where can I source it from so I can stop wasting time creating it manually?
I didn’t say that any “data” is boring. I said that building a streaming system is seemingly much more boring than playing around with procedural generation. Your data should be transform matrices, mesh grids and boxes.
Just start implementing a streaming system the best you can, and adjust as you go. You’ve “sourced” enough at this point 
1 Like
For working with the chunk stream system, you have several types of data. These can also be organised into sub types.
e.g.
asset types : mesh, scene
usage: vegetation, architecture, terrain scenery, characters, game mechanics,
If there are any enhancements please feel free to mention them.
Now vegetation can be managed entirely by the terrain engine. Most of the terrain engine addons have solutions for object scattering. For example, Terrain3D can scatter objects, or particles. All you need is a good control texture to mask out rocks and other areas, like deserts or water. The grass and small plants can often be implemented as a single mesh and not a scene file.
Trees are a type of vegetation, they should be billboards from a distance, so no more expensive than rendering cross quad grass. From close up they can be very expensive to render, so care must be taken in choosing or authoring the asset. There is a long story about trees.
Architecture is probably best handled with scene files that contain occluders, meshes and other data like lighting and maybe particle systems.
The architectural model might also have an interior - but there is no point in loading up a castle or space station interior from a great distance away if the player isnt going to see all that or interact any time soon. So you need to load the interiors and possibly some aspects of the exterior in chunks depending on when they will be visible.
Terrain scenery is similar to architectural exterior meshes.
Characters can be both CPU and GPU intensive, so they need to be loaded when they can interact or become visible - characters might need several LODs.
Scenes with Game mechanics can be loaded when the player could potentially interact with them.
When is the player close enough to a position to see or potentially interact with the object? This varies from type to type, and the loading time also varies for each data load. The system should log the time required to load data and adjust the timings according to performance. The player speed is trivially used to compute the time to reach an interaction point, then the data size is used to scale a loading bubble surrounding the object that causes the load to begin when the player enters. The larger scenes can be tweaked by their AABB screen size and the same time to load zone effect.
Going to cut the information here because i am starting to waffle about stuff I havent even implemented.
2 Likes
This will just overwhelm the OP. Most of this can be added later in the implementation process. For start just make a system that streams flat terrain patches and boxes from a database.
1 Like
I mentioned in an earlier post that I had a look at the Open World Database addon (OWDB). I was able to get the DB working in the editor but not in the game … and also it doesnt seem to use threads like the infinite terrain addons.
Its actually quite a useful technology, i would like to adapt it to my own system that uses a tile type grid for spacial hashing. Maybe I can hack through the script and replace it with my own code and keep the feature that loads only the local region in the editor …
I believe we are talking about different approaches.
I am referring to streaming data directly without relying on traditional scene management.
This involves using SQL (specifically SQLite) as the database, implemented in C++—including components like rendering_server.cpp—and handling data like this:
void SpatialStreamingManager::_consume_ready_chunks() {
LocalVector<BuiltChunk> local;
{
MutexLock lk(_ready_queue_mutex);
local = std::move(_ready_queue);
_ready_queue.clear();
}
}
Do you have an example template extension with some code? Does the Terrain3D code have examples of all the required data connections?
Without standard scene management you lose the ability to attach scripts to objects, they no longer call _process(delta), and they are not clipped in the BVH.
What happens when you add .res data into empty scene nodes? Can that be faster than modifying the scene tree, or is the slowdown of adding a chunk caused by the BVH and collision shape rebuild?
1 Like
So, maybe there are goals after all? Anything that can be done in two or more ways can be optimized.
Asking questions can help obtain new ideas — it’s called brainstorming.