-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Manually update model repository index #7173
Comments
I believe this is similar to: #7066 Is that accurate? |
Here is a potential workaround - though not finalized behavior: https://github.com/triton-inference-server/core/pull/340/files |
@nnshah1 Thank you for the response. I don't think that other issue is quite the same as what I'm describing. I'm not trying to upload the model in the load_model call, but rather I'm adding models to the model repository separately (not with triton client) and then I try to load them with triton client like this:
But this results in an error |
got it - I think I understand the steps here are:
--> error that it is unknown name? Can you confirm what version of Triton you are using / etc.? |
Yes that is correct
I am using |
@nnshah1 is there a way to configure triton to refresh the model index every 3 seconds? |
there is model control mode poll - which polls for changes - but are you wanting to "poll but not load" as we are discussing here? |
Is your feature request related to a problem? Please describe.
As I understand, when using
--model-control-mode=explicit
Triton server updates the model repository index only when first starting the server. If you add models with new names to the repository after the server has been started, the server doesn't recognize that they exist and trying to load them fails.Describe the solution you'd like
I'd like there to be an API call that allows you to update the repository index in case there are changes
Describe alternatives you've considered
I guess control mode poll regularly updates the index, but it also tries to load all models in the repository, which isn't ideal.
The text was updated successfully, but these errors were encountered: