Adding a new model?

[ak3ra]

What would be the process of adding a new pre-trained model to this inference pipeline?
I am curious about activity recognition or video classification models like videomae

[bwsw]

Hello @ak3ra, thank you for your question.

Let me clarify the question. Are you talking about a specific pipeline? Because Savant is a computer vision framework, not a pipeline. I assume that you may be talking about a certain sample... Clarify what sample you are talking about, please.

Aside of that...

Do you believe the model can run in real-time or faster than real-time? Savant is for models that can efficiently utilize Nvidia TensorRT engine.

For those models, it is beneficial to deploy them on optimized technologies, enabling optimization of the transactional costs between CPU RAM and GPU RAM. That is what Savant and DeepStream are about. If the model cannot be compiled to TensorRT and runs on PyTorch+CUDA with speed, say, 1 FPS or 0.1 FPS, there is no performance benefit from Savant or DeepStream.

Technically, there are no obstacles even running the inference with PyTorch inside Savant's pipeline. Previously, we developed pipelines using non-optimized technologies and optimized ones in a single pipeline, like object detection + EasyOCR for letter recognition. It works normally.

If you could help us understand the reasoning and motivation behind your question, we can prioritize the work to show how to use it. But, technically, the framework has everything required.