High ram usage procedural map generation

:information_source: Attention Topic was automatically imported from the old Question2Answer platform.
:bust_in_silhouette: Asked By TanguyBusschaert

Hi !

I’m new to Godot, and to game development in general ( already tried some projects in the past but didn’t stick to them )

I’d like to first of all say how of a nice community this is, a really positive and non toxic one, and that’s appreciated.

To the problem now, I’m a software engineer so don’t be afraid to throw stuff at me :slight_smile:

I’m generating a fixed size procedural map using simplex noise, for this example I’ll be generating a 2500 x 2500 map, tiles are 32 x 32 pixels.

The problem is I have quite a big of a RAM usage ( or maybe it’s normal for those dimensions ?), as I’m hitting around the 1.5GB RAM when running the game. As it’s the start of my game, so my core foundation, I’d like to already optimize this a little bit.

I’ll just explain how I got to this point as it isn’t much yet.

The first thing I did was importing the textures, just by putting them in the folder, re-importing them without a filter. Then I made a scene (2D Node), added each texture as a sprite, no collisions, just a sprite with the texture in it, then I exported that scene to a tile set resource.

Assets & TileSet picture

After that, I made my noise generator script which just fills and array with heights for all my x and y values.

extends Node2D

class_name NoiseGenerator

static func generate(width: int, height: int, mapSeed: int, octaves: int, period: int, lacunarity: float, persistence: float) -> Array:
	var generator = OpenSimplexNoise.new()
	
	generator.seed = mapSeed
	generator.octaves = octaves
	generator.period = period
	generator.lacunarity = lacunarity
	generator.persistence = persistence
	
	var heightMap: Array = []
	heightMap.resize(height * width)
	
	for y in height:
		for x in width:
			heightMap[y * width + x] = generator.get_noise_2d(x, y)
			
	return heightMap

After that I created my “Terrain” tilemap, which consists of the base layer of my map. It takes my heightmap, width & height as parameters and just matches the right tile based on the height ( the reason I match it in the dictionary as ‘heightValue’: “index”, is to be able to re-use the same tile on a different height value, let’s say I want to make grass plateaus on a mountain, the dictionary can’t have duplicate keys with different values so I use the boundary as key & the tile index as value.)

extends TileMap

class_name Terrain

enum TILES {
	None = -1
	DeepWater, 
	Water, 
	Sand, 
	Grass,
	LightDirt,
	Dirt,
	Snow,
}

const tile_heights = {
	-0.75: TILES.DeepWater, 
	-0.25: TILES.Water,
	-0.2: TILES.Sand,
	0.35: TILES.Grass,
	0.4: TILES.LightDirt,
	0.8: TILES.Dirt,
	1.0: TILES.Snow,
}

func generate_terrain(width: int, height: int, height_map: Array) -> void:
	for y in height:
		for x in width:
			var cell_height = height_map[y * width + x]
			var terrain = get_terrain_type(cell_height)
			set_cell(x, y, terrain)

func get_terrain_type(height: float) -> int:
	for tile_height in tile_heights:
		if height <= tile_height:
			return tile_heights[tile_height]
	return TILES.None

And finally I have a map scene where I will manage the different layers of my map.

extends Node2D

class_name Map

const MAP_WIDTH = 2000
const MAP_HEIGHT = 2000
const NATURE_NOISE_OCTAVES = 4
const TERRAIN_NOISE_OCTAVES = 8
const NOISE_PERIOD = 16
const NOISE_LACUNARITY = 0.6
const NOISE_PERSITENCE = 2

func _ready() -> void:
	randomize()
	var mapSeed = randi()
	
	var terrain_height_map = $NoiseGenerator.generate(MAP_WIDTH, MAP_HEIGHT, mapSeed, TERRAIN_NOISE_OCTAVES, NOISE_PERIOD, NOISE_LACUNARITY, NOISE_PERSITENCE)
	$Terrain.generate_terrain(MAP_WIDTH, MAP_HEIGHT, terrain_height_map)
	
	var nature_height_map = $NoiseGenerator.generate(MAP_WIDTH, MAP_HEIGHT, mapSeed, NATURE_NOISE_OCTAVES, NOISE_PERIOD, NOISE_LACUNARITY, NOISE_PERSITENCE)
	$Nature.generate_nature(MAP_WIDTH, MAP_HEIGHT, nature_height_map, terrain_height_map)

As you can see, I go trough this process again but for my “Nature” layer, which consists of a height map of the same seed but with more octaves, and based on those values + the values of the terrain map ( still need to figure a way out to get a generic way of doing this, but that’s another question ), I generate for example trees as for now. This part is not important as I had the RAM usage “issue” before implementing this part.

The result is good, I’ll add a picture of it too, but it’s just the RAM usage that concerns me, I hope you guys can figure something out, and sorry for the long post !

Example of generation ( the brown dots are my temporary tree textures :slight_smile: )

Example picture

Thank you !

I am a 3D guy. So my Tilemap experience is nil.

I just had a brief look at the code of tilemap implementation and the cell structure doesn’t look like it wastes a lot of memory. (seems like probably 16bytes per cell, 32bytes at most depending on align). Sure, there’ll be some more overhead to it but anyway…

So you’ve got roughly 6 million cells. Maybe you advance step by step first check what memory your array initialisation code alone requires. Also try to replace get_noise_2d() with a constant value to check if that function has some “leak”.

Then you could try to fiddle with the Tilemap. I.e. see if changing/increasing the quadrant size changes something.

wombatstampede | 2019-04-16 09:04

After reflection, I think it’s normal actually. Like you said, a cell is let’s say 16 bytes. My sprite texture is around 160 bytes so loading 1 cell into memory is 176 bytes. I tested it with a map of 2.500 x 2.500 which is 6.250.000 cells * 176 bytes which is 1.100.000.000 bytes or 1.1 GB, this plus some overhead of the editor it explains the usage.

I thought it trough, and I should not keep the whole map fixed in memory, I’m going to do the following: I’ll calculate the noise map & textures of a chunk, let say 64 by 64 tiles. I’ll setup a 3 by 3 grid of chunks, so 9 chunks, with my camera being in the middle one. When I move to let’s say the right chunk, i clear the most left chunks on my grid, shift all the chunks to the left and calculate a the new right side chunks.

This will give me an infinite seamless world, while only having 9 chunks of 64 x 64, which gives me 36.864 cells in memory instead of + 6 millions.

I’ll implement it when my break hits at work and post the solution.

TanguyBusschaert | 2019-04-16 09:50

Textures should be re-used by the tilemap. Anyway you’re right that things might add up quite fast with 6 million tiles.

In 3D worlds splatmap shaders are often used for such wide areas. That is basically a big index texture and multiple detail textures. But this is probably not appropriate here. Also new method, new challenges (unwanted interpolation, precision issues on mobile shaders, etc.).

wombatstampede | 2019-04-16 10:42

ive had a map that was 8000x8000 and it wasn’t 1.5gigs. it was around 310mb. use the tilemap node to make your maps :). the scene itself was around 1.2mb though.

shmellyorc | 2019-05-06 23:42

hey @shmellyorc. I know it’s been over two years, but do you think you could tell/show me how you achieved that? I’m getting over 400 mb with just 1000x1000. Any help would be appreciated. Thank you!

SpudBud | 2022-01-11 00:52