[Docs] How to test the demo?

[Elvis-tan]

Branch

main branch https://mmdetection3d.readthedocs.io/en/latest/

📚 The doc issue

In your demo, config_path = '../config/detection/embodied-det3d_8xb1_embodiedscan-3d-284class-9dof-mlvl.py'
checkpoint_path = '../ckpt/continuous.pth' were used, however, I didn't find these two files in your repository.

I used 'configs/detection/mv-det3d_8xb4_embodiedscan-3d-284class-9dof.py' , 'mv-3ddet.pth' provided by you, and your simple dataset office to test demo.ipynb. but I can only get one picture of the results, the length of results in results = model.test_step(collate_data) is 1.

And then I used 'configs/detection/cont-det3d_8xb1_embodiedscan-3d-284class-9dof.py' , 'cont-3ddet.pth' provided by you, but it crashed in nms_filter function, hope you could give more detailed guidance about how to test your demo, thanks a lot !

Suggest a potential alternative/fix

No response

[Elvis-tan]

I replaced some places in demo.py, and now I can run the demo with
'configs/detection/mv-det3d_8xb4_embodiedscan-3d-284class-9dof.py' , 'mv-3ddet.pth' .

for all for i in range(len(results)) , I changed them to for i in range(1, n_frames):

Specifically ,
for i in range(len(results)): image_ann = info['images'][selected_image[i]]
was changed to
for i in range(1, len(selected_image)): image_ann = info['images'][selected_image[i]]

Hope there will be a official response.

[mxh1999]

The 'configs/detection/cont-det3d_8xb1_embodiedscan-3d-284class-9dof.py' and 'cont-3ddet.pth' you used are correct.
Could you please provide more information about the crash in nms_filter?

[Elvis-tan]

The 'configs/detection/cont-det3d_8xb1_embodiedscan-3d-284class-9dof.py' and 'cont-3ddet.pth' you used are correct. Could you please provide more information about the crash in nms_filter?

As I mentioned above, I changed some places in your code, now I can run them without error, however, the output is not good as the original output in your demo.ipynb.

I used:
config_path = '../configs/detection/cont-det3d_8xb1_embodiedscan-3d-284class-9dof.py'
checkpoint_path = '../work_dirs/cont-3ddet/cont-3ddet.pth'

All I changed were as follows:

for all for i in range(len(results)) , I changed them to for i in range(1, n_frames):

Specifically , for i in range(len(results)): image_ann = info['images'][selected_image[i]] was changed to for i in range(1, len(selected_image)): image_ann = info['images'][selected_image[i]]

And some of my output are as follows:

Hope you could give some advices, thanks a lot.

[mxh1999]

From the places you changed, I understand that the results are not correct, even its length.
I can't reproduce this problem but I guess it may be due to incorrect inputs to the model.
Here are the sample inputs and outputs of the model. I hope this helps.
If you find the cause of this problem, please reply and let me know, thank you very much!

[Elvis-tan]

Does the demo in your repository only support 3d detection task for now? If not, then how could we test the capabilities of occupancy and visual grouding tasks in the wild?
Thanks a lot for your answer.

[mxh1999]

Does the demo in your repository only support 3d detection task for now? If not, then how could we test the capabilities of occupancy and visual grouding tasks in the wild? Thanks a lot for your answer.

Currently the demo only supports the 3d detection task, support for the occupancy and visual grouding tasks will be released after the CVPR 3D grounding challenge.