Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose onnxruntime session options (intra_op_num_threads) #2558

Open
Goktug opened this issue Apr 11, 2024 · 0 comments
Open

Expose onnxruntime session options (intra_op_num_threads) #2558

Goktug opened this issue Apr 11, 2024 · 0 comments

Comments

@Goktug
Copy link

Goktug commented Apr 11, 2024

The intra_op_num_threads setting determines the number of threads used for parallelizing the execution within nodes. By default, this value is set to 0, allowing onnxruntime to choose the appropriate number. However, as mentioned in microsoft/onnxruntime#8313, manually setting the number of threads may be necessary to avoid issues related to thread affinity.

When I use the library in a GPU machine with docker image, I get the following logs:

Executing node 273, title: Face Analysis Models, class type: FaceAnalysisModels
�[1;31m2024-04-08 08:26:07.167015003 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1129, index: 0, mask: {48, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2024-04-08 08:26:07.167027853 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1130, index: 1, mask: {1, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
�[1;31m2024-04-08 08:26:07.168290961 [E:onnxruntime:Default, env.cc:254 ThreadMain] pthread_setaffinity_np failed for thread: 1144, index: 15, mask: {8, }, error code: 22 error msg: Invalid argument. Specify the number of threads explicitly so the affinity is not set.�[m
...

To address this, I suggest adding onnxruntime session options to the FaceAnalysis class, which will allow users to manually adjust the relevant options based on their specific hardware setup.

I also found that another library fixed the same issue: dlr-eoc/ukis-csmask#20

Please let me know if there is already way to set intra_op_num_threads, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant