Graphics Tracker

[NHodgesVFX]

What problem does this solve or what need does it fill?

I looked through the issues and didn't see any open issues that would give an large overview of needed missing 3D graphics features in Bevy. So, I though I would create this tracker that could remain open and have items ticked of as they are added. Some features have multiple implementations that are possible, although it would be nice to have them all, I think it would be good to start with the easiest to implement in most cases. I am also sure I am forgetting some needed features as well, feel free to add them in the comments or if you have permission add them directly to this post. If this type of issue isnt wanted please feel free to close it :)

What solution would you like?

Rendering Methods

• Forward Clustered [Merged by Bors] - Clustered forward rendering #3153
• Deferred Deferred Renderer #9258

Reflections

• Initial Screen Space Reflections (SSR) in deferred renderer Implement opt-in sharp screen-space reflections for the deferred renderer. #12959
• Screen Space Reflections with roughness
• Screen Space Reflections Optimizations (Half-res tracing, etc...)
• Forward Renderer Screen Space Reflections
• Planar Reflections
• (HDRI) Environment Reflections, See Image Based Lighting and HDRI Support #4581
• HDRI Pre-Filtering Built into Bevy GenerateEnvironmentMapLight #9414
• Reflection Probes Implement minimal reflection probes (fixed macOS, iOS, and Android). #11366
• Reflection Probe Capturing
• Raytraced Reflections

Materials

• Support Refraction StandardMaterial Light Transmission #8015
• Support KHR_materials_volume
• Support KHR_materials_transmission StandardMaterial Light Transmission #8015
• Support KHR_materials_ior StandardMaterial Light Transmission #8015
• SubSurface Scattering (SSS)
• Alpha to Coverage Implement alpha to coverage (A2C) support. #12970
• Parallax Occlusion Mapping With Shadowing (POM) Add parallax mapping to bevy PBR #5928
• Tessellation and Displacement
• Vertex Colors [Merged by Bors] - Add support for vertex colors #4528
• Decals Custom projection / decal support #2401

Import

• Initial Auto Mip Map Generation
• Auto Mip Map Generation
• Auto UV Generation
• Auto Normal Generation Support calculating normals for indexed meshes #3987 Support calculating normals for indexed meshes #11654
• Auto Tangent Generation [Merged by Bors] - Generate vertex tangents using mikktspace #3872
• Acsii Embedded GTLF Support
• Texture Compression

Occlusion

• Screen Space Ambient Occlusion (SSAO) Screen Space Ambient Occlusion #3632
• Ground Truth Ambient Occlusion (GTAO) Screen Space Ambient Occlusion (SSAO) MVP #7402
• RayTraced Ambient Occlusion (RTAO)
• Bent Normals
• Approximate indirect specular occlusion Approximate indirect specular occlusion #11152

Global Illumination

• Screen Space Global Illumination (SSGI)
• Voxel Global Illumination (VXGI)
• SDF Global Illumination (SDFGI)
• Lightmaps Implement lightmaps. #10231
• Directional Lightmaps
• Lightmap Baking
• Light Probes Implement minimal reflection probes (fixed macOS, iOS, and Android). #11366 Implement irradiance volumes. #10268
• Irradiance Volumes Implement irradiance volumes. #10268
• Light Probe "Baking"
• RayTraced Global Illumination (RTGI)

Lights

• Spot Lights [Merged by Bors] - Spotlights #4715
• Rect Area Light
• Tube or Cylindrical Area Light
• Image Based Lighting, See Image Based Lighting and HDRI Support #4581

Shadows

• Cascaded Shadow Mapping Cascaded Shadow Maps #3629
• Sample-Distribution Shadow Maps Sample-Distribution Shadow Maps #4253
• Percentage Closer Filtering (PCF) Percentage Closer Filtering for shadows #3628 PCF For DirectionalLight/SpotLight Shadows #8006
• PCF for point lights Implement percentage-closer filtering (PCF) for point lights. #12910
• Percentage Closer Soft Shadows (PCSS) Percentage Closer Soft Shadows #3631
• RayTraced Shadows
• Screen Space Shadows (Contact Shadows)

Effects

• Fast approximate anti-aliasing (FXAA) [Merged by Bors] - Add FXAA postprocessing #6393
• Subpixel Morphological Antialiasing (SMAA) SMAA (Sub-pixel morphological antialiasing) support #9819
• Temporal Anti-Aliasing (TAA) Temporal Anti-Aliasing #3663
• Bloom [Merged by Bors] - Bloom #6397
• Tone Mapping [Merged by Bors] - separate tonemapping and upscaling passes #3425
• Depth Of Field
• Color Correction
• Color Grading
• Volumetric Lighting
• Depth Fog And Atmospheric [Merged by Bors] - Add Distance and Atmospheric Fog support #6412
• Auto Exposure / Eye Adaptation Implement Auto Exposure plugin #12792
• Upscaling - DLSS, XeSS, FSR 1, 2, & 3
• Dithering [Merged by Bors] - Fix color banding by dithering image before quantization #5264
• Robust Constrast Adaptive Sharpening (RCAS)
• Atmosphere / Sky rendering

Animation

• GPU Skinning
• Skeletal
• Morph Targets
• Animation Tracking Issue: MVP Animation #4026

Optimizations

• Occlusion Culling
• Hardware Instancing GPU Instancing #89
• Level Of Detail (LOD) Mesh LOD Support #6868
• Hierarchical Level of Detail (HLOD) Implement visibility ranges, also known as hierarchical levels of detail (HLODs). #12916
• Asset Streaming Add Support for Asset Streaming #8530
• Compute Shader Skinning Compute Shader Skinning #7502
• GPU-driven Rendering
• Virtual Textures
• Shadow Atlases
• Virtual Shadow Maps
• Caching Shadow Maps
• Automatic batching/instancing of draw commands Automatic batching/instancing of draw commands #9685

2 & 2.5D

• Lighting & Shading
• Global Illumination

Additional context

Refraction

• https://www.khronos.org/news/press/new-gltf-extensions-raise-the-bar-on-3d-asset-visual-realism

[alice-i-cecile]

This is an excellent list. I'll defer to @superdump, our rendering lead, for the best format for this.

You may also be interested in the rendering project board, which is a more operational view of the work that's being done.

[NHodgesVFX]

Thanks for the board its really interesting, I had found some of those issues just by searching but their are quite a few I didn't see before.

[alice-i-cecile]

If you have the expertise @NHodgesVFX, PR reviews from the community are incredibly valuable to help contributors learn and enable us to merge in features faster. This is especially true in rendering, where the team's expertise can often run a bit thin.

[NHodgesVFX]

Thanks, However I don't really have the expertise, I'm pretty new to rust and graphics programming. But maybe I will have the expertise sometime in the future :)

[james7132]

We're already skinning on the GPU. Unless the suggestion is to use bindless rendering to skin meshes, we're already doing this.

Animation blending and morph target work is already being tracked by #4026.

[NHodgesVFX]

I updated the list to reflect that, didn't realize skinning was already done on the GPU.

[cart]

Great list! I do think this feels more like Github Discussion territory rather than formal tracking issue territory. "3D graphics features" is a nebulous, large, and ever-changing area that we will never really be able to drive down to zero and close out. Any objections to me converting this to a discussion? You can still use this to track progress, but issues should be "scoped, well defined, and with clear paths to resolution".

[alice-i-cecile]

Yeah, I like this as a discussion too :)

[superdump]

A bunch of these have open issues and even PRs. I think they are all on the rendering project board. Could you add links to them all to the original post? I can’t edit it so I can’t maintain it either, whereas the project board is something I do keep on top of.

[NHodgesVFX]

Sure Ill add the links I see, no worries Ill try to keep it up to date as best I can.

[kirawi]

Are any of these good first issues?

[alice-i-cecile]

Unlikely, although if you have rendering expertise they'd probably be manageable :) #rendering-dev on Discord is quite helpful and does a good job teaching new contributors the ropes; I expect they'll have suggestions.

[superdump]

There's now a PR for spotlights: #4715

[JMS55]

For tonemapping, auto-exposure by calculating the luminance of each frame would be neat.

[NHodgesVFX]

Good call, added.

[JMS55]

To upscalers/external sdks, we should add FSR, XeSS, and DLSS (if the license allows us).

[minecrawler]

• FidelityFX-FSR is available as source using MIT
• FidelityFX-FSR2 is available as source using MIT
• XeSS is available as SDK using Intel Simplified Software License, which looks like it should be compatible
• DLSS 2 is available as SDK using MIT
• DLSS 3 is not available, yet, but it may become part of the Streamline SDK under MIT

[minecrawler]

To be honest, I would love to see a basic implementation, which works everywhere, in Bevy (for example custom or FSR), and have the others (FSR1/2, XeSS, DLSS2/3) available as plugins.

[NHodgesVFX]

Great Suggestions, I added them to the list.

[JMS55]

FXAA has been merged in #6393, and Bloom has been merged in #6397 🎉.

[NHodgesVFX]

Thanks for letting me know :) Updated the post

[matt328]

I'm curious about a plan for tessellation and displacement, with wgpu only supporting the basic pipeline stages, would things that use tessellation control and eval / hull and domain shaders need a different rendering library altogether? Wgpu seems like a great abstraction over lower level rendering APIs like OpenGL, Vulkan and DirectX, but I think the lack of support for tessellation and/or mesh shaders maybe become somewhat limiting. I'm admittedly not familiar with bevy's roadmap or future plans, so take all this with a grain of salt.

[NHodgesVFX]

You can do tessellation with a compute shader, which WGPU supports.

https://onrendering.com/data/papers/isubd/isubd.pdf

Also WGPU might add a extension for tesselation
gfx-rs/wgpu#222

[JMS55]

Cascaded shadows maps are now complete. We should also add Sample-Distribution Shadow Maps #4253 to the list.

[NHodgesVFX]

Thanks, I updated the tracker.

[wdhwg001]

Perhaps we could have a section mentioning that we have Clustered Forward Rendering (#3153 ) as the only one renderer.

And some other updates:

GPU instancing is tracked in #89

Animations are tracked in #4026

Dithering is finished in #5264

Distance and Atmospheric Fog is finished in #6412

Tonemapping is done and made into a separated mod in #3425

Parallax Mapping has a draft in #5928

[NHodgesVFX]

Thanks, Updated!

[JMS55]

Deferred: #9258
HDRI Pre-Filtering Built into Bevy: #9414
Ground Truth Ambient Occlusion (GTAO): Covered by the original SSAO PR

[NHodgesVFX]

Thanks Updated!

[JMS55]

For shadows, we should add screen space shadows, sometimes called contact shadows.

[NHodgesVFX]

Thanks for the suggestion, added!

[JMS55]

Under optimization, please add :

• Compute shader skinning Compute Shader Skinning #7502
• A general "GPU-driven rendering" task (no dedicated tracking issue, but there's some thoughts in GPU Instancing #89 (comment). Probably just don't link an issue for now).
• Asset streaming (meshes, materials, and textures), I suppose linking to Add Support for Asset Streaming #8530, but rendering needs special considerations tied into the culling systems.
• Shadow atlases/virtual shadow maps, and/or caching shadowmaps (these are all kind of linked)
• Level of detail Mesh LOD Support #6868

Under import, please add: Texture compression - now that Assets V2 has been merged, we have the ability to have asset metadata and preprocess assets. We should take material textures and preprocess them into astc/etc2/bcn/etc.

Under materials, please add decals #2401

Under effects, please add "Atmosphere/sky rendering"

[PlexusDuMenton]

You could also add Virtual Textures since you are talking about Virtual Shadow map

[NHodgesVFX]

Thanks everyone for the suggestions they have been added.

[samoylovfp]

Could Mipmap generation and texture compression be ticked off?

https://bevyengine.org/news/bevy-0-12/#compressedimagesaver

Bevy 0.12 ships with a barebones CompressedImageSaver that writes images to Basis Universal (a GPU-friendly image interchange format) and generates mipmaps.

[alice-i-cecile]

I wouldn't call that fully complete. I'd want a more automated and battle-tested mip-map generation process first.

[NHodgesVFX]

Thanks for the suggestions. I checked off texture compression and added a new task, which is checked off, for the initial implementation of MIP maps.

[PlexusDuMenton]

In addition, Deferred and Support Refraction should be ticked, (and likely "Support KHR_materials_transmission" and "Support KHR_materials_ior" but I didn't checked properly if they are finished)

Edit: PCF shadow for Directional & spotlight is also done (missing point lights)

[NHodgesVFX]

Thanks, Updated!

[mrgzi]

I don't know much about 3D graphic concept. Maybe this question is not related with this discussion but I am currious that can Bevy compete with Unreal Engine graphics when this list completed totally?

[mrgzi]

Thanks for your answer. Also, why there is no 2D graphics tracker?

[NHodgesVFX]

Advanced rendering features are less needed in 2D then in 3D. With 2D, most games rely on art and tricks, or custom solutions rather then advanced rendering. That being said 2D lighting and shading would be nice to see and of course there are other features important to 2D like batching, which was implemented recently. However, this is a good point I might rename this discussion to Graphics Tracker and add a 2D section.

For the most part in Bevy its possible to make pretty much any 2D game.

[mrgzi]

It would be great if you can also add 2D section. Maybe also 2.5D section?

[NHodgesVFX]

Thanks for the suggestion I added a combined 2 and 2.5D section to the tracker. Also, sorry it took a while to add. Also, renamed the tracker to Graphics Tracker.

[wdhwg001]

@NHodgesVFX Thanks for adding the 2D section! I was wondering if we could have 2D skeletal animation on the tracker list.

Also, the 2D section probably needs to have a Pixel Perfect feature, which is not built in but included in an example: #8112

[TirushOne]

@NHodgesVFX non directional lightmaps recentley got merged so you should probobley tick that one off

[NHodgesVFX]

Thanks for letting me know, Sorry for the wait, the tracker has been updated.

[pcwalton]

Lightmaps landed in #10231. Reflection probes and light probes landed in #11366. Voxel global illumination (VXGI) is #10268 and is on the milestone for 0.13.

[NHodgesVFX]

Thanks, Updated!

[pcwalton]

Auto exposure is submitted as a PR here: #12792

PCF for point lights landed in: d59b1e7

LOD support equivalent to Unity's is available as a PR: #12916
There's still more that could be done, however.

[NHodgesVFX]

Thanks, Updated!

[pcwalton]

Alpha to coverage is #12970

Basic screen space reflections are #12959

[NHodgesVFX]

Thanks, Updated!

[pcwalton]

1141e73 -- Alpha to Coverage -- is landed now

[pcwalton]

HLOD has landed #12916

Color grading has landed (and color correction too? not sure what you consider the difference to be) #13121

Auto exposure has landed #12792