Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate the Model and View Matrices #5287

Closed
1 task done
JetStarBlues opened this issue Jun 1, 2021 · 6 comments · Fixed by #6761
Closed
1 task done

Separate the Model and View Matrices #5287

JetStarBlues opened this issue Jun 1, 2021 · 6 comments · Fixed by #6761
Labels
Area:WebGL Enhancement
Projects
p5.js WebGL Projects

Comments

@JetStarBlues
Copy link
Contributor

JetStarBlues commented Jun 1, 2021 •
edited

Most appropriate sub-area of p5.js?

  • WebGL

Feature enhancement details:

Currently in the source code, the model and view matrices are combined into one (uMVMatrix). Typically, these two matrices are kept distinct from each other. The model matrix used only for translating / rotating / scaling the model. The view matrix used only for translating / rotating the camera.

One consequence of this, is that the model matrix is not available for use in shader programs. Currently, the view matrix and modelView matrix are passed to shaders as follows:

p5.Shader.prototype._setMatrixUniforms = function() {
  ...
  this.setUniform('uViewMatrix', this._renderer._curCamera.cameraMatrix.mat4);
  this.setUniform('uModelViewMatrix', this._renderer.uMVMatrix.mat4);
  ...
};

If the two are kept separate in the code, then providing the model matrix to shader programs becomes trivial.

p5.Shader.prototype._setMatrixUniforms = function() {
  ...
  const modelMatrix = this._renderer.modelMatrix;  // does not currently exist
  const viewMatrix = this._renderer._curCamera.cameraMatrix;
  const modelViewMatrix = (modelMatrix.copy()).mult(viewMatrix);

  this.setUniform('uModelMatrix', modelMatrix.mat4);
  this.setUniform('uViewMatrix', viewMatrix.mat4);
  this.setUniform('uModelViewMatrix', modelViewMatrix.mat4);
  ...
};

The separation of concerns will also make maintaining and/or expanding functionality in the source code simpler.

How would this new feature help increase access to p5.js?

Exposing the model matrix as a uniform available for users to use in their shader programs can make p5 a better platform for people looking to learn about 3D graphics and shaders. It will make it possible to use p5 to follow along with the wider ecosystem of tutorials, books, and other educational material on computer graphics.

---

This issue is an extension of the conversation here.
@stalgiag
Copy link
Contributor

stalgiag commented Jun 6, 2021

I agree with this. I found it painful to work on AR for p5.xr without separated model and view matrices.

As a side note, I think that this would set us up to allow for more of an object-based approach to manipulating 3D objects. I have longed for the day that we could just do myBox.position(x, y, z);. I find this more intuitive. In my experience new coders working in 2D with p5 begin by using the drawing functions that indicate position in the function (circle(x,y,r)). Some of the WebGL functions work this way but many don't and instead rely on translate(). The compound effects of translation functions can be confusing at first.

Anyways, I am a bit off-topic here but separated model and view matrices would open up the door to thinking about differently scoped model matrices as well.

@davepagurek davepagurek mentioned this issue Jan 14, 2023
Closed
17 tasks
@davepagurek davepagurek added this to System Level Changes in p5.js WebGL Projects Jan 16, 2023
@BarneyWhiteman
Copy link

Unfortunately I don't think I have the technical knowledge to implement this, nor the familiarity with the P5 code base, but I'd just like to throw my support behind this feature as well. It would be really useful to get the world-space position of vertices inside the vertex shader for certain effects.

@deveshidwivedi
Copy link
Contributor

Hello! I am trying to work on this. These are the changes made so far. I'd be grateful for any tips or suggestions to make things work. Thanks!

I request @davepagurek to please take a look.. Thank you!

@davepagurek
Copy link
Contributor

Thanks for taking this on, it's looking good! I noticed a few little things to update:

  • In src/webgl/p5.Camera.js it looks like you've updated it to use this.modelMatrix and this.viewMatrix in one spot. I think we don't need to update the names of the properties on the current camera (this.cameraMatrix and this.projMatrix still make sense to keep); it's just anything it sets on the renderer that might need to change. The nice thing about separating the model/view matrices is that now, when you switch cameras, you don't need to change the renderer's model matrix at all, you just need to replace its view matrix with the new camera's view matrix!
  • When setting a framebuffer, after updating the renderer's view matrix, I believe you need to reset the renderer's model matrix to an identity matrix in order to reset it. I think calling target.resetMatrix() will do that.
  • In resetMatrix, I'm not 100% sure but I think calling .set() with no args isn't supported yet as a way to reset it to an identity matrix. Maybe you could add a .reset() method to p5.Matrix that sets it back to the identiy matrix values found in the p5.Matrix constructor?
  • When setting uNormalMatrix, I think you want to do the inverse transpose on modelViewMatrix to take the camera transform into account too.

Feel free to open a PR for easier reviewing/conversation!

@deveshidwivedi
Copy link
Contributor

Sure, thank you!

@deveshidwivedi deveshidwivedi mentioned this issue Jan 21, 2024
Merged
3 tasks
@rheaney rheaney mentioned this issue Jan 30, 2024
Open
@nakednous
Copy link
Contributor

I've implemented a temporary hack for this using bindMatrices (source code available here). Check out the visualization of perspective transformation to NDC for an example. I'm also looking for advice on applying the shader there to scene lines in addition to just polygons?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area:WebGL Enhancement
Projects
p5.js WebGL Projects
Status: System Level Changes
p5.js WebGL Projects
  
System Level Changes
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Separate Model and View Matrices. deveshidwivedi/p5.js
7 participants
@nakednous @JetStarBlues @davepagurek @stalgiag @Qianqianye @BarneyWhiteman @deveshidwivedi