How can I add an object at the spot where the user taps?

[faizan1990]

This might sound like a simple question, but unfortunately it isn't (at least not in ViroReact and other similar technologies). Is there a way using model viewer to be able to add an object (as simple as a text label) where user taps in the scene / object?

If there's an example code where this is illustrated, that would be great.

[cdata]

@faizan1990 you're right to suggest that this is problem can seem deceptively simple. It's actually a pretty tricky thing to do (and even more so to do well).

The short answer is that this behavior is not currently supported by <model-viewer>, but we have plans to add support for similar features in the future, including:

• Support simultaneous viewing of multiple models #482 Support for simultaneous viewing of multiple models
• Support annotations a la Sketchfab's model viewer #334 Support for annotations

We want to define a model for rich, user-interactive composed scenes of 3D models that would work along the lines of what you are describing. But, we have a lot of other work to do first before we get there.

[faizan1990]

Hi @cdata, Thanks for your reply. Is there a way I can use in conjunction with ThreeJS to achieve the same? If not, is there a workaround?

[cdata]

Depending on your comfort level, there are various things you can do.

The most technically wholesome thing to do would be to craft your own component based on this one, and bundle it yourself. This would require some working knowledge of the Node.js/NPM tooling ecosystem.

If you want a quick / easy hack, another thing you could do is look up the Symbol that we use to hold the scene. This is considered private API, and there is no promise that it will be stable over time. The hack looks something like this:

const modelViewer = document.querySelector('model-viewer');
const $scene = Object.getOwnPropertySymbols(modelViewer)
     .filter(symbol => symbol.description === 'scene')[0];
const scene = modelViewer[$scene];

// scene is a Three.js Scene instance that contains the loaded model

With access to the scene, you should be able to do anything you would normally do with Three.js (including raycasting into the scene, adding objects etc.).

DISCLAIMER: The above hack involves diving into private API that could change without notice. Also, Three.js is considered a private implementation detail; at some time in the future we might decide to use completely different renderers under some circumstances, at which time this would no longer work as expected.

[faizan1990]

Hi @cdata, When I am console.loging the scene variable, I get a big glTF JSON object, but what I try to print / validate that glTF (at https://gltf-viewer.donmccurdy.com), it says that the version is below 2.0, whereas the glTF object I am using in model-viewer is definitely >= 2.0

THREE.GLTFLoader: Unsupported asset. glTF versions >=2.0 are supported. Use LegacyGLTFLoader instead.

Any idea why is it reporting version mismatch error?

[cdata]

@faizan1990 without a test case to look it, it's hard to speculate. Can you throw something up on Glitch, perhaps?

[faizan1990]

@cdata I am using the code in this example: https://googlewebcomponents.github.io/model-viewer/examples/augmented-reality.html. Upon running your workaround in the browser developer tools, I get the value of scene variable as a big blob of JSON (which appears to me as a glTF object).

I am saving that JSON as a .gltf file and when I upload that file to https://gltf-viewer.donmccurdy.com, it's giving me the version error:

THREE.GLTFLoader: Unsupported asset. glTF versions >=2.0 are supported. Use LegacyGLTFLoader instead.

Am I missing any step?

[marwie]

Depending on your comfort level, there are various things you can do.

The most technically wholesome thing to do would be to craft your own component based on this one, and bundle it yourself. This would require some working knowledge of the Node.js/NPM tooling ecosystem.

If you want a quick / easy hack, another thing you could do is look up the Symbol that we use to hold the scene. This is considered private API, and there is no promise that it will be stable over time. The hack looks something like this:

const modelViewer = document.querySelector('model-viewer');
const $scene = Object.getOwnPropertySymbols(modelViewer)
     .filter(symbol => symbol.description === 'scene')[0];
const scene = modelViewer[$scene];

// scene is a Three.js Scene instance that contains the loaded model

With access to the scene, you should be able to do anything you would normally do with Three.js (including raycasting into the scene, adding objects etc.).

DISCLAIMER: The above hack involves diving into private API that could change without notice. Also, Three.js is considered a private implementation detail; at some time in the future we might decide to use completely different renderers under some circumstances, at which time this would no longer work as expected.

Hello, I've modified the scene slightly and doing raycasts, it seems though as if the modelviewer only re-renders once the camera orbits or user input is detected. So click and raycasts change e.g. something which is not immediately reflected, is there a way to tell modelviewer to re-render / set the scene dirty?

I've found the symbol "needsRender" but I dont know how/if I can trigger that? Setting the scene dirty didnt help.
https://github.com/GoogleWebComponents/model-viewer/blob/24158a11e0d21f6e127a17f9b692f3f73a123335/src/model-viewer-base.ts

[cdata]

@marwie it should be sufficient to invoke modelViewer[$needsRender](), but you need to do it on every frame.

[marwie]

thanks for the quick reply. I've tried getting the needsRender symbol already like so (but cant seem to get it)

$needsRender = Object.getOwnPropertySymbols(modelViewer).filter(
    symbol => symbol.description === "needsRender"
  )[0];

for example I'm setting the camera target in the threejs update loop but changes are only reflected when I start dragging in modelviewer. e.g.

function update() { requestAnimationFrame(update); t += 0.04; cameraTarget.y += Math.sin(t) * .001; }

(camera target is the vector3 from camera controls)

[cdata]

You can just set scene.isDirty = true on each frame. That's all the [$needsRender] method does anyway.

[marwie]

Ok so I guess it has to do with how CameraControls are updated / passed to update the actual camera. Thanks