-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TensorRT 8.6.3.1 package in Python PyPy for Triton Nvidia Inference Server version > 24.01 #7221
Comments
@mc-nv can you take a look? |
Triton is a part of NVIDIA Optimized Framework It's not possible to change subset of the libraries release with NVIDIA Optimized Framework on demand. We understand the request. If you want to use TensorRT 8.6.1 within latest container please feel free to modify image per own needs. |
thanks! @mc-nv can you help me - how to install in triton image "tensorrt==8.6.1.6"? I am trying to use this guide (https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-861/install-guide/index.html#installing-debian), but I am getting errors like:
|
Description
For Triton Nvidia Inference Server version bigger than
24.01
(started with24.02
) the supported version of tensorrt is8.6.3.1
. I am using tensorrt python package and script to convertonnx
weights totrt
engine, but the last available version in pypy is8.6.1.6
and because of this I can't usetensorrt_backend
in triton and got this error:Is it possible to upload this package version (
8.6.3.1
) to pypy? Or how can I rewrite this script using other tools?Triton Information
24.02-24.04, docker image (without additional building)
To Reproduce
pip install tensorrt==8.6.1.6
Linked issue
Expected behavior
I can use tensorrt backend with correct version
The text was updated successfully, but these errors were encountered: