Having problems making Compositor Effects

Help Shaders
, ,
1

Godot Version

4.6 stable mono

Question

So, I’m trying to make a datamoshing shader using compositor effects, but I’m running into two problems now.

First, what is the correct way to pass one buffer to the next frame? I tried making an image, but that isn’t working too well, it’s expensive and very prone to not working 100% of the time, for some reason. Just passing one RID to the next frame clears the buffer, so I’m at a loss.

Second, how do I get the camera view matrix in glsl inside compute? The documentation is super scarce and google didn’t help. I couldn’t find a way to pass it as a push constant.

I really want to make a tutorial on this sort of thing, just so there is documentation around for other people struggling with the same things, but I can’t solve them myself…

2

Make a custom texture on the rendering device and read/write from/to it.

You’ll need to send the data via an uniform buffer, or if you don’t have much you can put it into a push constant. You can get common matrices from within _render_callback() through the render_data argument.

3

I was doing prevtex[view] = rd.texture_create_shared(RDTextureView.new(), prevtex[view]). I assume a shared texture is bad? But idk how to use the rd.texture_create()…

I can see render_data.get_camera_attributes() function, but how do I manipulate the data it provides? It’s just an RID, not the camera itself…

4

Create a new texture using create_texture(), set it as an uniform and read/write to that.

render_data.get_render_scene_data() will return RenderSceneData object that you can query. You’ll still need to build your own uniform buffer though. Most of that data appears to already be packed into an existing uniform buffer. You can get it by calling RenderSceneData.get_uniform_buffer() but I couldn’t find how the buffer data is actually structured, anywhere in the documentation.

5

That’s my problem with this whole project. There is no documentation or tutorials on this. I read the docs, there’s nothing there.

There’s no documentation, what’s an RDTextureFormat and do I need to create a new one using .new(), or do I need to know to get one preexisting from somewhere to correctly make a texture???

Man, there’s so little information about these things, I really want to make a tutorial when this is all done. No one should jump through this many hoops to do something that’s relatively simple.

6

You can just load some dummy texture resource of the wanted size, and fetch its RD RID from the RenderingSever.

And for camera matrix, pack it into your own buffer with whatever structure you want.

1 Like
7

That already helps, but do you know how to do that to make a simple mockup example?

I really can’t find anything on my own, I’ve read all I could find, and nothing explains how, only sometimes it explains what.

8

RDTextureFormat is straightforward. Yes, you need to new() a new object and populate it with data, most of it is self explanatory if you know the basics of dealing with bitmaps. So as a first step, try to create a texture, fill it with a color and let the compositor effect display it. You should be able to do that with the compositor effect boilerplate from the docs.

9

At this point, I’ll write out what I currently have and ask for more specific help.

var rd : RenderingDevice
var shader : RID
var main_pipeline : RID
var reset := 0
var nearest_sampler

var prevtex : Array[RID] = []

func _render_callback(_effect_callback_type: int, render_data: RenderData) -> void:
	if not rd or not shader or not main_pipeline:
		initialize_cs()
		return
	
	var scene_buffers : RenderSceneBuffersRD = render_data.get_render_scene_buffers()
	if not scene_buffers: return
	
	var size : Vector2i = scene_buffers.get_internal_size()
	if size.x == 0 or size.y == 0: return
	#var cam = render_data.get_camera_attributes()
			
	var x_groups : int = (size.x - 1) / 8 + 1
	var y_groups : int = (size.y - 1) / 8 + 1
	
	var push_constants : PackedFloat32Array = PackedFloat32Array()
	push_constants.append(size.x)
	push_constants.append(size.y)
	push_constants.append(Time.get_ticks_msec())
	push_constants.append(reset)
	
	for view in scene_buffers.get_view_count():
		if not rd or not shader or not main_pipeline:
		initialize_cs()
		return
	
	var scene_buffers : RenderSceneBuffersRD = render_data.get_render_scene_buffers()
	if not scene_buffers: return
	
	var size : Vector2i = scene_buffers.get_internal_size()
	if size.x == 0 or size.y == 0: return
	var cam = render_data.get_camera_attributes()
	
	var fmt = RDTextureFormat.new()
	fmt.width = size.x
	fmt.height = size.y
	fmt.format = RenderingDevice.DATA_FORMAT_R16G16B16A16_SFLOAT
	fmt.usage_bits = RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT \
					| RenderingDevice.TEXTURE_USAGE_STORAGE_BIT \
					| RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT \
					| RenderingDevice.TEXTURE_USAGE_CPU_READ_BIT \
					| RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT
		#for i in scene_buffers.get_view_count():
			#prevtex.append(RDTextureFormat.new())
			#prevtex[i].width = size.x
			#prevtex[i].height = size.y
			#prevtex[i].format = RenderingDevice.DATA_FORMAT_R16G16B16A16_SFLOAT
	
			#prevtex[i] = rd.texture_create(prevtex[i], RDTextureView.new(), [Image.create(size.x, size.y, false, Image.FORMAT_RGBA16).get_data()])
			
	var x_groups : int = (size.x - 1) / 8 + 1
	var y_groups : int = (size.y - 1) / 8 + 1
	
	var push_constants : PackedFloat32Array = PackedFloat32Array()
	push_constants.append(size.x)
	push_constants.append(size.y)
	push_constants.append(Time.get_ticks_msec())
	push_constants.append(reset)
	
	for view in scene_buffers.get_view_count():
		var screentex : RID = scene_buffers.get_color_layer(view)
		var motiontex : RID = scene_buffers.get_velocity_layer(view)
		var depthext := scene_buffers.get_depth_layer(view)
		#var prevframe : Texture2DRD = Texture2DRD.new()
		#prevtex[view].usage_bits = RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT | RenderingDevice.TEXTURE_USAGE_STORAGE_BIT | RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT
		if prevtex.size() <= view:
			prevtex.append(rd.texture_create(fmt, RDTextureView.new(), [rd.texture_get_data(screentex, 0)]))
		
		#prevframe.texture_rd_rid = prevtex[view]
		var masktex := ShaderGlobals.mask_tex
		var maskdepth := ShaderGlobals.depth_tex
		
		#print(ShaderGlobals.auxtex)
		
		var uniform_screen : RDUniform = RDUniform.new()
		uniform_screen.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
		uniform_screen.binding = 0
		uniform_screen.add_id(screentex)
		
		var uniform_vector := RDUniform.new()
		uniform_vector.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
		uniform_vector.binding = 1
		uniform_vector.add_id(motiontex)
		
		var uniform_depth := RDUniform.new()
		uniform_depth.uniform_type = RenderingDevice.UNIFORM_TYPE_SAMPLER_WITH_TEXTURE
		uniform_depth.binding = 2
		uniform_depth.add_id(nearest_sampler)
		uniform_depth.add_id(depthext)
		
		var uniform_prev := RDUniform.new()
		uniform_prev.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
		uniform_prev.binding = 3
		#prevtex[view] = (scene_buffers.get_color_layer(view))
		print(prevtex[view])
		uniform_prev.add_id(prevtex[view])
		
		var uniform_mask := RDUniform.new()
		uniform_mask.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
		uniform_mask.binding = 4
		uniform_mask.add_id(masktex)
		
		var uniform_maskdepth := RDUniform.new()
		uniform_maskdepth.uniform_type = RenderingDevice.UNIFORM_TYPE_SAMPLER_WITH_TEXTURE
		uniform_maskdepth.binding = 5
		uniform_maskdepth.add_id(nearest_sampler)
		uniform_maskdepth.add_id(maskdepth)
		
		var image_uniform_set : RID
		image_uniform_set = UniformSetCacheRD.get_cache(shader, 0, [uniform_screen, uniform_vector, uniform_depth, uniform_prev, uniform_mask, uniform_maskdepth])
		
		var compute_list : int = rd.compute_list_begin()
		
		rd.compute_list_bind_compute_pipeline(compute_list, main_pipeline)
		rd.compute_list_bind_uniform_set(compute_list, image_uniform_set, 0)
		rd.compute_list_set_push_constant(compute_list, push_constants.to_byte_array(), push_constants.size() * 4)
		rd.compute_list_dispatch(compute_list, x_groups, y_groups, 1)
		rd.compute_list_end()
		
		#prevtex[view] = rd.texture_create(RDTextureView.new(), prevtex[view])
		#prevtex.append(RDTextureFormat.new())
		#prevtex[view].width = size.x
		#prevtex[view].height = size.y
		#prevtex[view].format = RenderingDevice.DATA_FORMAT_R16G16B16A16_SFLOAT
		#prevtex[view].usage_bits = ReneringDevice.TEXTURE_USAGE_CAN_UPDATE_BIT | RenderingDevice.TEXTURE_USAGE_STORAGE_BIT | RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT
		
		prevtex[view] = rd.texture_create(fmt, RDTextureView.new(), [rd.texture_get_data(prevtex[view], 0)])
		#prevtex[view] = prevframe.texture_rd_rid
		if reset == 0:
			reset += 1

Assume the omitted code is near 1:1 to the boilerplate code on the official docs.

Is the texture create being done correctly? I have a feeling I should configure what RDTextureView.new() is, but I have no idea how.

I wish there was a simple instruction on how to do this on docs, so I could read there instead of bothering people on a sunday, but alas.

Edit: I think I figured out how to get the texture, but now its RID is 0?

10

Umm you don’t want to create a new texture at every frame. That would be insane, and you’ll quiclky get out of vram.

Using the default values in RDTextureView is fine for typical use cases.

1 Like
11

Here’s a simple example that may help you. It creates a single pixel texture on the main rendering device and displays it on a sprite:

extends Sprite2D

func _ready():
	# get rendering device
	var rd: RenderingDevice = RenderingServer.get_rendering_device()
	
	# setup texture format object
	var format: RDTextureFormat = RDTextureFormat.new()
	format.width = 1
	format.height = 1
	format.usage_bits = RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT | RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT | RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT
	format.format = RenderingDevice.DATA_FORMAT_R8G8B8A8_UNORM
	
	# create texture on the rendering device
	var texture_rid: RID = rd.texture_create(format, RDTextureView.new(), [[255, 255, 0, 255]])
	
	# create a gdscript texture object for the rd texture so we can assign it to a aprite
	texture = Texture2DRD.new()
	texture.texture_rd_rid = texture_rid
	scale = Vector2(20, 20)

image
image141×132 108 Bytes

If you have problems figuring out what to put into the format object, just get the format object from the rendering device for one of the comparable textures created by the engine, and print out what’s in there.

1 Like
12

Still, I don’t need to display it, I just need to preserve it for exactly one frame, so it can be used on the next one, cumulatively.

Datamoshing is an effect that, if simulated, would use the motion vectors of the scene, and apply them to a cumulative nonclearing frame. Hard to explain, but relatively easy to implement.

If I create a texture every frame, but free_rid() it before assigning a new one, that could work, right?

13

You don’t need to display it but you should display it as a debugging strategy to check that everything went well with its creation. The code demonstrates basic texture creation. How can I show that it works other than displaying it?

Instead of creating a texture You can render into a viewport and use that texture in the compositor.

Or you can create a single texture at startup and read/write to it at each frame.

No, don’t create a texture every frame. It’s an expensive operation. Besides what would be the point of freeing and then re-creating it if you already have one?

1 Like
14

It took me waaay to long to realize you can create a texture with an empty array as a valid parameter. What I needed to do was create a texture at _init(), then use it as an RID, and it worked. Debug texture is for debugging purposes. I rewrote it from the beginning.

@tool
class_name FromScratch extends CompositorEffect

var prevtex : Array[RID] = []

func _init() -> void:
	...
	prevtex.clear()
	
	fmt = RDTextureFormat.new()
	fmt.width = size.x
	fmt.height = size.y
	fmt.format = RenderingDevice.DATA_FORMAT_R16G16B16A16_SFLOAT
	fmt.usage_bits = RenderingDevice.TEXTURE_USAGE_CAN_UPDATE_BIT \
					| RenderingDevice.TEXTURE_USAGE_STORAGE_BIT \
					| RenderingDevice.TEXTURE_USAGE_CAN_COPY_FROM_BIT \
					| RenderingDevice.TEXTURE_USAGE_CPU_READ_BIT \
					| RenderingDevice.TEXTURE_USAGE_SAMPLING_BIT
	
	prevtex.append(rd.texture_create(fmt, RDTextureView.new(), []))
	print(prevtex[0])
	ShaderGlobals.debug_tex = Texture2DRD.new()
	ShaderGlobals.debug_tex.texture_rd_rid = prevtex[0]

func _render_callback(_effect_callback_type: int, render_data: RenderData) -> void:
	...
	var scene_buffers : RenderSceneBuffersRD = render_data.get_render_scene_buffers()
	if not scene_buffers: return
	
	size = scene_buffers.get_internal_size()
	if size.x == 0 or size.y == 0: return
	fmt.width = size.x
	fmt.height = size.y
	
	var x_groups : int = (size.x - 1) / 8 + 1
	var y_groups : int = (size.y - 1) / 8 + 1
	
	var push_constants : PackedFloat32Array = PackedFloat32Array()
	push_constants.append(size.x)
	push_constants.append(size.y)
	push_constants.append(Time.get_ticks_msec())
	push_constants.append(reset)
	
	for view in scene_buffers.get_view_count():
		...
		if prevtex[view].get_id() == 0:
			prevtex[view] = ShaderGlobals.debug_tex.texture_rd_rid
			ShaderGlobals.debug_tex.texture_rd_rid = prevtex[view]
		...
		var uniform_prev := RDUniform.new()
		uniform_prev.uniform_type = RenderingDevice.UNIFORM_TYPE_IMAGE
		uniform_prev.binding = 3
		uniform_prev.add_id(prevtex[view])
		...
		var image_uniform_set : RID
		image_uniform_set = UniformSetCacheRD.get_cache(shader, 0, [uniform_screen, uniform_vector, uniform_depth, uniform_prev, uniform_mask, uniform_maskdepth])
		
		var compute_list : int = rd.compute_list_begin()
		
		rd.compute_list_bind_compute_pipeline(compute_list, main_pipeline)
		rd.compute_list_bind_uniform_set(compute_list, image_uniform_set, 0)
		rd.compute_list_set_push_constant(compute_list, push_constants.to_byte_array(), push_constants.size() * 4)
		rd.compute_list_dispatch(compute_list, x_groups, y_groups, 1)
		rd.compute_list_end()
		
		#print(rd.texture_get_data(prevtex[view], 0)) # This made it run at 1 frame per 5 minutes, it was fun.
		
		if reset == 0:
			reset += 1
1 Like