Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to use triton client with shared memory in C++ (Jetpack 6 device) #7177

Open
ganeshmojow opened this issue May 1, 2024 · 4 comments
Labels
module: platforms Issues related to platforms, hardware, and support matrix

Comments

@ganeshmojow
Copy link

Cross-posted issue from Triton Client repository.
Description
I am using the tritonserver + client igpu release (tritonserver2.41.0-igpu.tar.gz) on a jetpack 6 device. I want to use the shared memory functions with the triton client which are declared in shm_utils.h and defined in shm_utils.cc. However, the header is not found in Triton Client's include directory leading to a compilation error.

Triton Information
What version of Triton are you using?
Client Version: v2.41.0

Are you using the Triton container or did you build it yourself?
I am using the Triton Client igpu release.

To Reproduce
The import #include "shm_utils.h" fails.

Expected behavior
The import should be successful and the shared memory functionality should be available.

Note:
On making the following changes to src/c++/library/CMakeLists.txt and building the client from source, I was able to import the header and use the shared memory functions. (Triton Client Branch - r23.12)

@@ -84,12 +84,12 @@ if(TRITON_ENABLE_CC_GRPC OR TRITON_ENABLE_PERF_ANALYZER)
   # libgrpcclient object build
   set(
       REQUEST_SRCS
-      grpc_client.cc common.cc
+      grpc_client.cc common.cc shm_utils.cc
   )
 
   set(
       REQUEST_HDRS
-      grpc_client.h common.h ipc.h
+      grpc_client.h common.h ipc.h shm_utils.h
   )
 
   add_library(
@@ -257,12 +257,12 @@ if(TRITON_ENABLE_CC_HTTP OR TRITON_ENABLE_PERF_ANALYZER)
   # libhttpclient object build
   set(
       REQUEST_SRCS
-      http_client.cc common.cc cencode.c
+      http_client.cc common.cc cencode.c shm_utils.cc
   )
 
   set(
       REQUEST_HDRS
-      http_client.h common.h ipc.h cencode.h
+      http_client.h common.h ipc.h cencode.h shm_utils.h
   )
 
   add_library(
@@ -394,6 +394,7 @@ if(TRITON_ENABLE_CC_HTTP OR TRITON_ENABLE_CC_GRPC OR TRITON_ENABLE_PERF_ANALYZER
       FILES
       ${CMAKE_CURRENT_SOURCE_DIR}/common.h
       ${CMAKE_CURRENT_SOURCE_DIR}/ipc.h
+      ${CMAKE_CURRENT_SOURCE_DIR}/shm_utils.h
       DESTINATION include
   )
 
@rmccorm4
Copy link
Collaborator

rmccorm4 commented May 2, 2024

Hi @ganeshmojow,

Thanks for filing an issue. @nv-kmcgill53 could you help take a look here?

@rmccorm4 rmccorm4 added the module: platforms Issues related to platforms, hardware, and support matrix label May 2, 2024
@ganeshmojow
Copy link
Author

Following up to check if there are any updates related to this issue.

@nv-kmcgill53
Copy link
Contributor

CC: @jbkyang-nvi

@ganeshmojow
Copy link
Author

Checking back to see if there are any updates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: platforms Issues related to platforms, hardware, and support matrix
Development

No branches or pull requests

3 participants