An Invitation to 3D Vision is an introductory tutorial on 3D vision (a.k.a. geometric vision or visual geometry or multi-view geometry). It aims to make beginners understand basic theory of 3D vision and implement their own applications using OpenCV. In addition to tutorial slides, example codes are provided in the purpose of education. They include simple but interesting and practical applications. The example codes are written as short as possible (mostly less than 100 lines) to be clear and easy to understand.
- Download tutorial slides
- Download example codes in a ZIP file
- Read how to run example codes
- Update new examples: fisheye un-distortion, Ceres Solver: tutorials and simple mpc controller
- fisheye un-distortion: [fisheye_undistortion.cpp]
- ceres_tutorial: [ceres_tut1, ceres_tut2, ceres_tut3]
- simple mpc controller: [ceres_mpc.cpp]
- The main title, An Invitation to 3D Vision, came from a legendary book by Yi Ma, Stefano Soatto, Jana Kosecka, and Shankar S. Sastry. We wish that our tutorial will be the first gentle invitation card for beginners to 3D vision and its applications.
- The subtitle, for everyone, was inspired from Prof. Kim's online lecture (in Korean). Our tutorial is also intended not only for students and researchers in academia, but also for hobbyists and developers in industries. We tried to describe important and typical problems and their solutions in OpenCV. We hope readers understand it easily without serious mathematical background.
- Single-view Geometry
- Camera Projection Model
- Object Localization and Measurement: object_localization.cpp (result: image)
- Image Formation: image_formation.cpp (result: image0, image1, image2, image3, image4)
- Geometric Distortion Correction: distortion_correction.cpp (result: video)
- General 2D-3D Geometry
- Camera Calibration: camera_calibration.cpp (result: text)
- Camera Pose Estimation (Chessboard): pose_estimation_chessboard.cpp (result: video)
- Camera Pose Estimation (Book): pose_estimation_book1.cpp
- Camera Pose Estimation and Calibration: pose_estimation_book2.cpp
- Camera Pose Estimation and Calibration w/o Initially Given Camera Parameters: pose_estimation_book3.cpp (result: video)
- Camera Projection Model
- Two-view Geometry
- Planar 2D-2D Geometry (Projective Geometry)
- Perspective Distortion Correction: perspective_correction.cpp (result: original, rectified)
- Planar Image Stitching: image_stitching.cpp (result: image)
- 2D Video Stabilization: video_stabilization.cpp (result: video)
- General 2D-2D Geometry (Epipolar Geometry)
- Visual Odometry (Monocular, Epipolar Version): vo_epipolar.cpp
- Triangulation (Two-view Reconstruction): triangulation.cpp
- Planar 2D-2D Geometry (Projective Geometry)
- Multi-view Geometry
- Bundle Adjustment
- Global Version: bundle_adjustment_global.cpp
- Incremental Version: bundle_adjustment_inc.cpp
- Structure-from-Motion
- Global SfM: sfm_global.cpp
- Incremental SfM: sfm_inc.cpp
- Feature-based Visual Odometry and SLAM
- Visual Odometry (Monocular, Epipolar Version): vo_epipolar.cpp
- Visual Odometry (Stereo Version)
- Visual Odometry (Monocular, PnP and BA Version)
- Visual SLAM (Monocular Version)
- Direct Visual Odometry and SLAM
- Visual Odometry (Monocular, Direct Version)
- c.f. The above examples need Ceres Solver for bundle adjustment.
- Bundle Adjustment
- Correspondence Problem
- Line Fitting with RANSAC: line_fitting_ransac.cpp
- Line Fitting with M-estimators: line_fitting_m_est.cpp
- Appendix
- Line Fitting
- Planar Homograph Estimation
- Fundamental Matrix Estimation
- OpenCV (> 3.0.0, 3-clause BSD License)
- OpenCV is a base of all example codes for basic computer vision algorithms, linear algebra, image/video manipulation, and GUI.
- Ceres Solver (3-clause BSD License): A numerical optimization library
- Ceres Solver is additionally used by m-estimator, bundle adjustment, structure-from-motion, and visual odometry/SLAM.
- You can see the same examples as above in the following script.
- We recommend running the examples in Anaconda.
It was tested in the following version.
- OpenCV (4.2.0v)
- Scipy (1.8.0v)
- SciPy (pronounced “Sigh Pie”) is an open-source software for mathematics, science, and engineering.
- Numpy (1.22.3v)
- The fundamental package for scientific computing with Python
- Open3D (0.13.0v)
- Open3D is an open-source library that supports rapid development of software that deals with 3D data.
The authors thank the following contributors and projects.
- Jae-Yeong Lee: He motivated many examples.
- Giseop Kim: He contributed the initial version of SfM codes with Toy-SfM and cvsba.
- The KITTI Vision Benchmark Suite: The KITTI odometry dataset #07 was used to demonstrate visual odometry and SLAM.
- Russell Hewett: His two hill images were used to demonstrate image stitching.
- Kang Li: His shaking CCTV video was used to demonstrate video stabilization.
- Richard Blais: His book cover and video in the OpenCV tutorial were used to demonstrate camera pose estimation and augmented reality.