Transformation process of a Python Pytorch GPU model into an optimized C++ ONNX TensorRT one.
Based on the Fruits 360 dataset, a model is trained with Pytorch GPU.
See the full process on:
- Notebook 1: Dataset preprocessing
- Notebook 2: Model creation
The PyTorch model is evaluated. It is then converted to an ONNX model and evaluated again to ensure that its conversion does not degrade its performance.
See the full process on:
- Notebook 3: Pytorch model evaluation
- Notebook 4: ONNX convertion and evaluation
The ONNX model is finally deployed on the GPU via TensorRT, where it is evaluated again.
See the full process on:
- Notebook 5: TensorRT in Python
C++ transcription follows the same process as notebook 5 "TensorRT in Python".
PyTorch to TensorRT via ONNX:
- Tutorial: PyTorch to Onnx TensorRT Python
- Tutorial: Onnx to TensorRT Python
- Example: Onnx to TensorRT C++
- Ubuntu
18.04 - CUDA
10.2 - libCuDNN
7.6.5(dev + runtimes) - TensorRT
7.0.0- Developper guide, Python API, C++ API
- OpenCV
4.5.1(lib + contrib) - C++ installation guide
The following package can be installed with pip install
- torch and torchvision
- opencv-python
- onnx
Other useful libraries
These very useful libraries are not used in this project, but could be interesting for data preprocessing.
- boost, with
sudo apt install libboost-all-dev- docs - hdf5, with
sudo apt-get install libhdf5-dev- C++ API, Read example - HighFive
2.2.2, with Cmake - Github, C++ API - xtl
0.7.2, with CMake - C++ API - xtensor
0.23.0, with Cmake - Github, C++ API, Tutorial - xtensor-io dependencies, with
sudo apt-get install libsndfile-dev libopenimageio-dev zlib1g-dev - xtensor-io
0.12.1, with Cmake - Github, C++ API