Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Gaussian Splats performance #6344

Open
cognitiveplus opened this issue May 8, 2024 · 3 comments
Open

Improve Gaussian Splats performance #6344

cognitiveplus opened this issue May 8, 2024 · 3 comments
Assignees
Labels
area: graphics Graphics related issue performance Relating to load times or frame rate

Comments

@cognitiveplus
Copy link

cognitiveplus commented May 8, 2024

Recently Adobe researchers published a paper called GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting.

The page contains real-time demos featuring this viewer: antimatter15 / splat.

What's interesting about it is that it produces solid 60fps on M1 Air even with advanced scenes, while the same PLY imported into PlayCanvas results in less FPS (with Device Pixel Ratio unchecked).

For example:
https://sai-bi.github.io/project/gs-lrm/viewer/index.html?is_object=false&url=https://huggingface.co/datasets/saibi/gs-lrm/resolve/main/scene_assets/002000_gaussians.ply
https://playcanv.as/index/7d0ab8f5 (https://playcanvas.com/project/1214533/)

From viewer description it looks like it's doing splat sorting on CPU in a separate worker thread on async basis:

the final approach that i settled on is to run the sorting process on the CPU in a webworker, which happens a bit more slowly (at roughly 4fps whereas the main render is at 60fps), but that's fine because most of the time when you are moving around the z order doesn't actually change very fast (this results in momentary artifacts when jumping directly between different camera orientations on opposite sides).

I guess PC might introduce similar configuration for 3DGS, with selector in inspector allowing to choose a sorting algo.

@mvaligursky
Copy link
Contributor

We're doing splat sorting on a separate thread, and that bit is pretty fast. - faster than 4fps for sure I think. 1M splats in about 4ms on M1 Max.
But we have other bottlenecks we're investigating currently.

@mvaligursky mvaligursky added performance Relating to load times or frame rate area: graphics Graphics related issue labels May 8, 2024
@cognitiveplus cognitiveplus changed the title Improve Gaussian Splats performance by sorting splats in a worker thread Improve Gaussian Splats performance May 8, 2024
@MAG-AdrianMeredith
Copy link

The sorting is definitely the slowest part in my experience, it causes a lot of hitching as the gpu upload locks rendering so the fps meter might be reporting a good fps but its dropping frames.
Its still awesome though, especially for a first release

@mvaligursky
Copy link
Contributor

related: #6352

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: graphics Graphics related issue performance Relating to load times or frame rate
Projects
None yet
Development

No branches or pull requests

4 participants