[Rendering] Overhaul Rendering Backend

[michaelsakharov]

At the moment Rendering is very basic, In Every frame the camera loops over all objects looking for renderable ones, and called the OnRenderObject method, which then immediately draws the object. This works fine but there's an issue, When we do this we have to loop over all objects in the frame for Update() and then again for each camera for OnRenderObject(), This also makes Sorting objects if they have a sorting order harder, and also the threading is practically out of the question.

So let's rewrite it to be more like this:

Frame Start
    Foreach GameObject call Update()
        On Renderables in Update() call something like Graphics.DrawMesh()
        DrawMesh() does not immediately draw but instead adds it to a Draw command list
        The command list can be a SortedList, so things can be sorted by an int
        Things can also be sorted in layers
    At the end of the frame Foreach Camera
        Ensure the previous frame has finished rendering (Block and wait)
        Push the previous frame results onto the camera Target
        Kick-off a new Thread and Draw the current frame Command List inside it
        Taking into account camera settings like what layers it can see

This method results in only looping over GameObjects once, then 1 loop over what actually rendered which is much less work, it also enables Sorting objects by some Order and layer, AND it comes with a Render thread :D

-Update 8/06/24
I've also decided to also strip out all the fancy rendering pipeline stuff, and revert to a simpler system closer to unity 4-5.
There are a few reasons, but the biggest is I don't want to spend time reinventing the wheel and creating a super fancy customizable node-based rendering system, only to find in 5 years it's not really that great. I would rather pull back and do something that we know works then after all the features we need are implemented, consider upgrading to something fancier.

I still plan todo the above with the Command List architecture since it lets us multi-thread rendering.

[PaperPrototype]

What about using the EngineObject.Foreach() method?

[PaperPrototype]

also will it be possible to have multiple transparent objects layered over eachother?

[PaperPrototype]

I know the guys who made TheMachinery game engine had layered transparency working flawlessly

[michaelsakharov]

What about using the EngineObject.Foreach() method?

I mean the engine just loops over a Dictionary holding all engine types, EngineObject.AllObjects is a DIctionary<Type, EngineObject>, I've generally just directly looped all inside that dictionary, because idk what components are renderable or anything, i cant specify a type i have to go over them all, and we go over them all to update anyway.
Like you could implement your own renderable component that doesnt inherit anything it just called Graphics.DrawMesh inside it randomly, that ideally should just work.

Also will it be possible to have multiple transparent objects layered over eachother?

I mean we already can, just need to order transparency to the end of rendering and with culling off.
But I'm considering switching to Forward rendering just for transparency which solves all the transparency issues, it's called Hybrid rendering, I want to expand the Node Graph to just be able to handle this automatically for us, so I don't explicitly have to implement hybrid rendering.

[michaelsakharov]

We could also have Light shadowmaps be passed through a render graph.
Ideally 100% of everything rendering related is passed through a render graph

[sinnwrig]

While looking at how the shader compilation pipeline works currently, I noticed that it locks the engine to GLSL and an OpenGL-only backend, since it relies on the device to compile a program and set uniforms by name. To improve the extensibility of the backends, it might be beneficial to bundle glslang/DXC and SPIRV-Cross with the engine. This way, the shader compilation pipeline could do the following:

• Shader code is compiled with glslang/DXC into SPIR-V bytecode.
• Output SPIR-V is cross-compiled into GLSL, HLSL, MSL, with SPIRV-Cross for relevant target graphics APIs.
• Cross-compilation results are serialized along with additional information pertaining to backend-specific data and uniform/attribute locations.
• The proper shader can then be provided to the rendering backend to use at runtime.

This approach decouples shader compilation from the graphics backend, since at serialization-time the engine can determine what backends are being targeted and cross-compile shaders for each one. This also simplifies implementing backends which don't allow you to get uniform and attribute locations by name, such as Vulkan. Instead, SPIRV-Cross can be used to perform reflection on the shaders and the binding locations can be bundled into the serialized shaders.

If this engine feature is useful and does not conflict with preexisting plans or internal development, I would be more than willing to put in the work to get this added in a pull request, although it would require some discussion.

A side note on compilers: I have a preexisting C# wrapper for glslang which could serve useful, and am currently working on creating a wrapper for SPIRV-Cross as well. Aside from the preexisting wrapper, I am biased towards recommending using glslang over DXC, as it is more lightweight, has a simpler build system, exposes a C-style interface, can compile both GLSL and HLSL shaders, does not depend on a closed-source DLL to be bundled with it, and is easier to get working on Mac.

[michaelsakharov]

I replied in Discord, However, I'll copy-paste it here.

I plan to work on this eventually. I've compiled shaders through SPIRV for Prowl before but removed them temporarily since I wasn't fully confident in my implementation.

I'm most familiar with OpenGL and can advance Prowl fastest with it. For now, I'll focus on OpenGL to get the engine to a usable state before diving into areas I'm less familiar with.

I'm open to pull requests for this and would hugely appreciate if someone else more knowledgeable on the topic could implement it.

My priority is to replace Unity as soon as possible for my own projects. So, I'm focusing on what I already know to progress quickly.

I would absolutely love it if you are able to put work toward this.
My biggest concern I had previously was I didn't know the extent of SPIRV's features of Glsl. Does it support everything Glsl has and if not what specific features don't work?

The project's code is super messy at the moment, unfortunately, so I apologize in advance haha.