Yolov8 in Unity

[cmilanes93]

Search before asking

• I have searched the YOLOv8 issues and discussions and found no similar questions.

Question

Hi, has any one try it Yolo v8 on mobile using unity and achive good FPS??

Additional

No response

[glenn-jocher]

Hi there! 🚀

Integrating YOLOv8 in Unity for mobile deployment isn't directly supported, but you can achieve it by exporting the model to ONNX and then utilizing a plugin or framework compatible with Unity for ONNX models, like [Unity Barracuda](https://docs.unity3d.com/Packages/com.unity.barracuda @cmilanes93).

Here’s a basic example of how you might export your model to ONNX:

yolo export model=yolov8n.pt format=onnx

Then, load this ONNX model into Unity with Barracuda. Ensure to optimize your model for mobile deployment to achieve good FPS, such as using smaller model variants and quantization if necessary.

Feel free to ask more if you need further guidance! 😊

[cmilanes93]

Hi there! 🚀

Integrating YOLOv8 in Unity for mobile deployment isn't directly supported, but you can achieve it by exporting the model to ONNX and then utilizing a plugin or framework compatible with Unity for ONNX models, like [Unity Barracuda](https://docs.unity3d.com/Packages/com.unity.barracuda @cmilanes93).

Here’s a basic example of how you might export your model to ONNX:

yolo export model=yolov8n.pt format=onnx

Then, load this ONNX model into Unity with Barracuda. Ensure to optimize your model for mobile deployment to achieve good FPS, such as using smaller model variants and quantization if necessary.

Feel free to ask more if you need further guidance! 😊

Quantization? We already build our mobile app with the yolov8, but we are trying to solve the problem of fps on mobile.
We retrain our model only using CPU, and the smaller version, but still having poor performance on cell.

[glenn-jocher]

Hi! Glad to hear you've made progress with YOLOv8 on mobile. If you're still facing low FPS issues, consider the following:

1. Model Quantization: Reduces the precision of the weights from floating point to int8, which can significantly improve performance without a large sacrifice in accuracy. ONNX provides tools for quantization that you can utilize.

2. Optimize the Inference Pipeline: Ensure that input preprocessing and output postprocessing are optimized. Reducing the resolution or simplifying preprocessing steps might help speed up the process.

3. Use Hardware Accelerators: If your mobile device supports it, make use of hardware accelerators like GPU or DSP.

Here's a quick snippet on how you might quantize in ONNX:

import onnx
from onnxruntime.quantization import quantize_dynamic, QuantType

model_path = "model.onnx"
quantized_model_path = "quantized_model.onnx"

model = onnx.load(model_path)
quantized_model = quantize_dynamic(model, quantized_model_path, weight_type=QuantType.QInt8)

onnx.save(quantized_model, quantized_model_path)

Implementing these can potentially help improve your app's performance. Let me know if this helps or if you need more details!