Alex/open3d ml tutorials #545

INF800 · 2022-06-16T06:31:39Z

Re-write condensed tutorials based on suggestions.

Changes were made to 3 tutorial notebooks

docs/tutorial/notebook/01_config_files.ipynb
docs/tutorial/notebook/02_datasets.ipynb
docs/tutorial/notebook/03_train_model.ipynb

Suggested changes were:

The notebooks must look similar to already existing open3d tutorial notebooks (all notebooks)
Run all the notebooks files and check if they work locally (all notebooks)
Working directory must be docs/tutorial/notebook/ for all data, model checkpoint logs etc. (all notebooks)
Remove garbage-in garbage-out dataset graphic. (01_config_files.ipynb)
Use consistent vector notations (01_config_files.ipynb)
Change to right-handed coordinate system (02_datasets.ipynb)
Change colour of axes in the coordinate system (02_datasets.ipynb)

Please note:

Some cells of 02_datasets.ipynb will not run because the notebook does not have access to smaller version of semanticKITTI dataset. ~~@sanskar107 can you share a downloadable link for that dataset that I can wget in the the notebook?~~ Will use shared link https://cdn.discordapp.com/attachments/903644960211992607/946792875423838288/semkitti.zip.
I will pass all the text information through gramarly before merging once structure and content gets green signal.
I will remove Inference_on_a_custom_data.ipynb, reading_a_config_file.ipynb, reading_a_dataset.ipynb, train_ss_model_using_pytorch.ipynb and train_ss_model_using_tensorflow.ipynb before merging because they are essentially duplicates. Keeping them here for reference temporarily.

This change is

+ Condense all notebooks into 3 notebook tutorials. + Replace SemanticKITTI with Toronto3D in training notebook (small size & fast training). + TODO: Pass through gramarly (may find multiple mistakes), please focus on content & structure.

review-notebook-app · 2022-06-16T06:31:43Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

ssheorey · 2022-06-17T04:05:01Z

docs/tutorial/notebook/01_config_files.ipynb

@@ -0,0 +1,910 @@
+{


You can access configuration values as object attributes (cfg.{property_name}) or dictionary key values (cfg['{property_name}'])

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/02_datasets.ipynb

@@ -0,0 +1,449 @@
+{


Remove Open3D-ML for consistency

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/02_datasets.ipynb

@@ -0,0 +1,449 @@
+{


Standard convention for object detection lidar datasets (e.g. KITTI / Waymo) is x(red) = forward, y (green) = left, z (blue) = up. Add XYZ labels to the axis.

Mention that datasets may instead of r (lidar reflectance) as feature, or no features at all.

A dataset is partitioned into train(ing), valid(ation) and test(ing) splits. Models use data from the training split to update network weights and performance is measured on the validation split. Hyperparameters can be adjusted to optimize performance on the validation split. The test split is never used during the training step and is only used to report the final performance.

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


Both PyTorch and TF can be installed. User code just needs to choose one inside a specific python file.
requirements-torch.txt if Nvidia GPU is not available.

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


This section should be in the datasets tutorial.

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


holdout -> test split for consistency

Reply via ReviewNB

ssheorey · 2022-06-17T04:05:02Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


Also talk about:
Continuing training from a checkpoint.
Point to the tensorboard tutorial for visualizing training progress.

Reply via ReviewNB

ssheorey · 2022-06-17T15:01:40Z

docs/tutorial/notebook/01_config_files.ipynb

@@ -0,0 +1,910 @@
+{


See Readme.md for how to import Config

Reply via ReviewNB

ssheorey · 2022-06-17T15:01:41Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


remove this listing

Reply via ReviewNB

ssheorey · 2022-06-17T15:01:41Z

docs/tutorial/notebook/03_train_model.ipynb

@@ -0,0 +1,583 @@
+{


Only import the first line and then use the imported namespace.

Reply via ReviewNB

INF800 added 3 commits June 2, 2022 14:16

update reading_a_config_file.ipynb

398c342

Re-write tutorial notebooks

cabba7e

+ Condense all notebooks into 3 notebook tutorials. + Replace SemanticKITTI with Toronto3D in training notebook (small size & fast training). + TODO: Pass through gramarly (may find multiple mistakes), please focus on content & structure.

re-write condensed tutorials based on suggestions

7aa40a8

INF800 requested review from sanskar107 and ssheorey June 16, 2022 06:32

ssheorey reviewed Jun 17, 2022

View reviewed changes

INF800 added 2 commits June 21, 2022 14:28

use _ml3d, re-write a cell in 01_config_files.ipynb

a80523e

add more content to md cells

06e9ae2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Alex/open3d ml tutorials #545

Alex/open3d ml tutorials #545

INF800 commented Jun 16, 2022 •

edited

review-notebook-app bot commented Jun 16, 2022

ssheorey Jun 17, 2022

INF800 Jun 21, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022 •

edited

INF800 Jun 21, 2022

ssheorey Jun 17, 2022

ssheorey Jun 17, 2022

Alex/open3d ml tutorials #545

Are you sure you want to change the base?

Alex/open3d ml tutorials #545

Conversation

INF800 commented Jun 16, 2022 • edited

Re-write condensed tutorials based on suggestions.

review-notebook-app bot commented Jun 16, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ssheorey Jun 17, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

INF800 commented Jun 16, 2022 •

edited

ssheorey Jun 17, 2022 •

edited