help/FAQ/

[giscus[bot]]

help/FAQ/

Find solutions to your common Ultralytics YOLO related queries. Learn about hardware requirements, fine-tuning YOLO models, conversion to ONNX/TensorFlow, and more.

https://docs.ultralytics.com/help/FAQ/

[SumitJana404]

suppose I am fine-tuning a yolov8 model with custom dataset, so can the fine tuned model detect the newly trained class images as well as pretrained class of images? If yes how to do that

[pderrenger]

@SumitJana404 yes, a fine-tuned YOLOv8 model can detect both the newly trained classes and the pre-trained classes. To achieve this, you need to include the data for both the new and pre-existing classes during the fine-tuning process. Here's a concise guide on how to do it:

1. Prepare Your Dataset: Combine your custom dataset with a subset of the pre-trained dataset that includes the classes you want to retain. Your dataset should have annotated images for both the new and pre-existing classes.

2. Create a Custom Data YAML: Make a YAML file that lists all the classes (new and pre-existing) and update the paths to your combined dataset.

3. Modify the Model Configuration: If necessary, adjust the model configuration to match the total number of classes (new + pre-existing).

4. Fine-Tune the Model: Start training with the train.py script, using the combined dataset and the pre-trained weights. Set the --weights argument to the pre-trained model file.

5. Validate the Model: After training, validate the model using val.py to ensure it performs well on both new and pre-existing classes.

Remember to balance your dataset so that the model is not biased towards the new or pre-existing classes. 🧐

For detailed steps and assistance, refer to the Ultralytics documentation. If you encounter any issues, feel free to open a GitHub issue in the YOLOv8 repo. Good luck with your fine-tuning! 🚀

[DishantSave]

How can i convert my trained YOLOv8 model to .tflite format while keeping the tensors intact like 'locations', 'classes', 'scores', 'number of detections' in the converted .tflite file?

[glenn-jocher]

@DishantSave yes, when fine-tuning a YOLOv8 model with a custom dataset, the fine-tuned model can detect both the newly trained classes and the pre-trained classes. To achieve this, you should include the pre-trained classes in your custom dataset and continue training (fine-tuning) from the weights of a pre-trained model. This way, the model retains knowledge of the pre-trained classes while learning to recognize new ones.

For converting your trained YOLOv8 model to the .tflite format, you can use the Export mode supported by YOLOv8. This mode allows you to convert the model to various deployment-ready formats, including TensorFlow Lite. During the export process, ensure that you specify the correct export format and follow the guidelines to maintain the output tensors' structure. For detailed instructions on exporting models, please refer to the Export section in our documentation.

[DishantSave]

Ok fine, Thank you!

[pderrenger]

@DishantSave you're welcome! If you have any more questions or need further assistance, feel free to ask. Good luck with your YOLOv8 model conversion! 👍

[dongsig2]

In YOLOv8, when training with the option 'Pretrained = True', is the pretrained dataset the COCO dataset? Or was it pretrained on a different dataset?

[pderrenger]

@dongsig2 in YOLOv8, when you use the option pretrained=True, it implies that the model will initialize with weights that have been pre-trained on a large dataset, typically the COCO dataset, which is a standard dataset used for object detection tasks. The COCO dataset is widely used due to its large size and variety of object classes, making it a good starting point for many object detection applications. If you're using one of the official YOLOv8 pre-trained models provided by Ultralytics, it's likely pre-trained on COCO. However, always check the specific model documentation or release notes for the exact details of the pre-training dataset.

[mehmetyalcin94]

Hello, i want to train the pretrained yolov8 detection model to detect only 1 class then use base yolo segmentation model to segment this trained object. This 1 class is involved in segmentation model classes so I am hoping I won't have to train the segmentation model. Is it possible?

So basically my problem is yolov8 detection model doesnt detect my object accurate enough but segmentation works amazing. Is there any other solution that I can't see?

[glenn-jocher]

Hello!

It's definitely possible to train the YOLOv8 detection model on a single class and then use the base YOLO segmentation model if the class is already included in the segmentation model's classes. You won't need to retrain the segmentation model in this case.

For the detection model not being accurate enough, you could try fine-tuning it on your specific class with more data or adjusting hyperparameters. This could help improve its detection accuracy before passing the detections to the segmentation model.

Here's a quick example of how to fine-tune your detection model for one class:

yolo train data=your_dataset.yaml model=yolov8n.pt --single-cls

Make sure your_dataset.yaml is configured correctly for your single class. If you need further assistance or have more questions, feel free to ask! 😊

[mehmetyalcin94]

Hello,

I completed my training and i have best.pt and last.pt outputs. How can i use this model on top of yolo segmentation model? This part I couldn't figure out. I am using python, it feels easier than CLI.

For my yaml file I have this, is it enough? Or do I need some extra configuration for single class training.

Classes

names:
0: first

[pderrenger]

Hey there!

Great job on completing your training! 🎉 To use your trained best.pt or last.pt model with the YOLO segmentation model, you can load the weights into the segmentation model in Python like this:

from ultralytics import YOLO

# Load your trained weights into the segmentation model
segmentation_model = YOLO('path/to/yolov8-segmentation-model.pt').load('path/to/best.pt')

Regarding your YAML file for single-class training, it looks good! Just ensure that the rest of the YAML file is correctly set up with the paths to your training and validation datasets.

If you run into any issues or have more questions, the community is here to help. Happy coding! 😊

[fninsiima]

I wonder about the documentation for the parameter iou under predict.
It says "Higher values result in fewer detections by eliminating overlapping boxes, useful for reducing duplicates."

I have tested this and this seems untrue. When I changed from the default of 0.7 to 0.9 I got more overlapping boxes. When I decreased it to 0.5 I got fewer overlapping boxes. I am wondering if there is an issue with the documentation or the backend.

[glenn-jocher]

Hey there! 👋 It looks like there might be a bit of confusion regarding the iou parameter. The iou parameter is indeed intended to reduce duplicates by eliminating overlapping boxes when set to a higher value. However, if you're experiencing the opposite, it could be due to specific characteristics of your dataset or the particular objects you're detecting.

In some cases, if objects are very close together, a higher iou might cause the model to consider them as a single object, hence the increase in overlapping boxes. Conversely, a lower iou can be more aggressive in separating detections, reducing overlaps.

Here's a quick example of how you might adjust the iou parameter during prediction:

from ultralytics import YOLO

# Load your model
model = YOLO('path/to/your/model.pt')

# Set the IOU threshold
iou_threshold = 0.5  # Lower value to reduce overlaps

# Run prediction with the adjusted IOU
results = model.predict('path/to/your/image.jpg', iou=iou_threshold)

If you continue to see unexpected behavior, it might be worth double-checking the implementation or reaching out with a detailed issue on the repo so we can take a closer look! 🕵️‍♂️

Hope this helps! Let us know if you have any more questions.

[dave-meagher]

Very simple question, but I can't find the answer anywhere :-) The in yolonv8 small model, the output tensor "Output0" has a last layer name of "/model.22/concat_5". The tensor output is the following "output0 (data type: Float_32; tensor dimension: [1, 84, 8400]; tensor type: APP_READ)"

Can you explain what is the data structure of the [1, 84, 8400] so I can decode the inference results I'm in C++ code.

[glenn-jocher]

Hey there! 😊 It looks like you're diving into the details of the YOLOv8 small model's output tensor structure. Let's break down the [1, 84, 8400] dimensions for you:

• The first dimension 1 represents the batch size, indicating that the output is for a single image.
• The second dimension 84 is the combined data for each detection, typically including:
  • The bounding box coordinates (4 values: x, y, width, height),
  • The objectness score (1 value),
  • The class probabilities (for YOLOv8 small, assuming 80 classes, this would be 80 values).
  • So, 4 + 1 + 80 = 85. However, if it's 84, you might want to check if there's a different configuration or a missing/extra value in the model's output specification.
• The third dimension 8400 represents the total number of detections across the grid cells.

To decode this in C++, you'll iterate through the 8400 detections, for each detection parsing out the bounding box, objectness score, and class probabilities from the 84 values.

Here's a very simplified example of how you might start to access these values:

float* output = get_model_output(); // This function is hypothetical
for(int i = 0; i < 8400; i++) {
    float* detection = output + (i * 84);
    // Now, `detection[0]` to `detection[3]` are your bbox coordinates,
    // `detection[4]` is your objectness score,
    // `detection[5]` to `detection[84]` are the class probabilities.
}

This is a basic approach and you'll need to adjust it based on the actual model output and your application needs. Hope this helps clear things up! If you have more questions, feel free to ask.

[dave-meagher]

Wow! Thank you! That is exactly what I was looking for. 💯 :-)

[glenn-jocher]

@dave-meagher i'm thrilled to hear that the information was exactly what you were looking for! 😊 If you have any more questions or dive deeper into customizing and optimizing your YOLO models, remember that the Ultralytics documentation and our community on GitHub are always here to help. Happy coding! 🚀

[Hateley123]

I am going through your training scripts but I am a little confused on the pything script. I see you create a new model with yolov8n.yaml and then you train the model using coco128.yaml. Is the yolov8n.yaml a file used to just initialize the model for the script to work and then your "custom" data set its the coco128.yaml?

[Hateley123]

Sorry. This was a dumb questions. You can ignore it. I cant seem to delete it though.

[pderrenger]

No worries at all! There are no dumb questions here, and we're happy to clarify any confusion. 😊

You're correct in your understanding. The yolov8n.yaml is indeed used to initialize the model's architecture, specifying layers, filters, and other architectural parameters. On the other hand, coco128.yaml is a dataset configuration file that tells the model where to find the training and validation data, how many classes there are, and other dataset-specific settings.

So, in short, yolov8n.yaml defines the model structure, and coco128.yaml points to your data for training. This setup allows you to easily switch between different models or datasets by simply changing the respective YAML files.

Don't worry about not being able to delete your question; it might help others with the same query! If you have any more questions or need further assistance, feel free to ask. Happy coding! 🚀

[aslakm]

I have trained a yolov8 model with images of road surface cracks. I have annotated the cracks with masks. And then I do prediction on model and saves the prediction results to a json file using prediction[0].tojson() function. When I compare the image I get with the plot() method with the images I plot based on the result in the json, file I observe that the predict result is different. This is the case if the image contains several cracks. But the bounding rectangle is correct. I have plotted points at the image based on the xy coordinates given in the Segments object of the json file. Why do I get different prediction result by using the predictions found in the json file?

[glenn-jocher]

Hey there! 👋 Thanks for reaching back out. It sounds like there might be an issue with how the segmentation masks are being generated or extracted in the lower part of your image. Since the upper part matches well but the lower part is missing, this could potentially be a bug or an edge case that wasn't caught during the mask generation process.

A quick suggestion would be to double-check the mask generation logic or the post-processing steps that might be filtering out or not correctly handling the lower part of the image. If you're using custom scripts to extract or plot these masks, ensure they're consistently applied across the entire image.

If everything seems in order on your end, it might be helpful for us to take a closer look. Could you please open an issue on the GitHub repository with details of your observation, including how you're processing the JSON output? This way, our team can investigate further and provide a more targeted solution. 🕵️‍♂️

Thanks for your patience and for bringing this to our attention!

[aslakm]

Hi again!
The imagesize is 2448x2048. If you have a look in the json output from Yolo, there is no y coordinates above 1100. I have now reported this as a bug on github.

Best
Aslak

[glenn-jocher]

@aslakm hi Aslak,

Thanks for bringing this to our attention! 🕵️ It's quite intriguing that the y coordinates in the JSON output don't go above 1100 for an image size of 2448x2048. This indeed sounds like something we need to investigate further.

As a temporary workaround, could you please ensure that the imgsz during prediction matches your original image size? Sometimes, discrepancies can arise if there's a mismatch. Here's a quick example of how to specify imgsz during prediction:

yolo detect predict model=yolov8n.pt source='your_image.jpg' imgsz=2448,2048

We'll dive into the bug report you've submitted and work on identifying the root cause. Your detailed observation will surely help us in pinpointing the issue faster.

[aslakm]

Hi Glenn!
I trained the model with image size of 640. The image i do the prediction on is 2448x2048. When I use imgsz=640, I still have the same problem. You will find my weight file at the link below.
https://drive.google.com/file/d/1e9jOTuXYpY4q9_qih_3YICHddyGM8Lda/view?usp=drive_link

[glenn-jocher]

@aslakm hey there! 👋 Thanks for sharing your model and the details of your setup. If the issue persists even with imgsz=640 during prediction on an image of size 2448x2048, it's possible that there's a discrepancy in how the model is handling different image sizes.

Given the size difference, here's something you might try to ensure consistency: use the same imgsz for both training and prediction. Since you trained with an imgsz of 640, predictions should theoretically align better when the input images are resized to the same dimensions. However, given the nature of your issue, there seems to be something else at play.

# Example prediction command
model.predict('path/to/image.jpg', imgsz=640)

Could you also check if your annotations/masks are correctly scaled or if there's any preprocessing step that might be affecting the results unexpectedly?

In the meantime, I'll take a look at your weight file and investigate further. Thanks for flagging this, and let's aim to resolve it soon! 🚀

[SumitJana404]

I want to perform classification using yolo-v8 and I want to classify object based on no. of boxes predicted on a image. If the model detect single bounding box then it is of class A and if it detects two bounding boxex then it belongs to class B. Is it possible, if yes then how

[glenn-jocher]

Hey there! 😊

Absolutely, you can classify objects based on the number of bounding boxes detected in an image using YOLOv8. Here's a simple approach:

1. Use YOLOv8 to detect objects in your image.
2. Count the number of bounding boxes (detections) returned by the model.
3. Based on the count, classify the image into Class A or B.

Here's a quick code snippet to get you started:

from ultralytics import YOLO

# Load your model
model = YOLO('yolov8n.pt')  # Adjust model path as needed

# Predict on an image
results = model('path/to/your/image.jpg')

# Count the number of detections
num_detections = len(results[0].boxes)

# Classify based on the number of detections
if num_detections == 1:
    print("Class A")
elif num_detections == 2:
    print("Class B")
else:
    print("Other Class")

This is a basic example. You might need to adjust it based on your specific requirements. If you have more complex classification criteria or need further assistance, feel free to ask! 🚀

Happy coding!

[aslakm]

I have used the same image size 640 for both training and prediction. Have you dowladed the weight file and run the test your self? Best Aslak Sendt fra Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ Fra: Glenn Jocher ***@***.***> Sendt: Friday, March 15, 2024 12:44:21 PM Til: ultralytics/ultralytics ***@***.***> Kopi: Aslak Myklatun ***@***.***>; Mention ***@***.***> Emne: Re: [ultralytics/ultralytics] help/FAQ/ (Discussion #7928) @aslakm<https://github.com/aslakm> hey there! 👋 Thanks for sharing your model and the details of your setup. If the issue persists even with imgsz=640 during prediction on an image of size 2448x2048, it's possible that there's a discrepancy in how the model is handling different image sizes. Given the size difference, here's something you might try to ensure consistency: use the same imgsz for both training and prediction. Since you trained with an imgsz of 640, predictions should theoretically align better when the input images are resized to the same dimensions. However, given the nature of your issue, there seems to be something else at play. # Example prediction command model.predict('path/to/image.jpg', imgsz=640) Could you also check if your annotations/masks are correctly scaled or if there's any preprocessing step that might be affecting the results unexpectedly? In the meantime, I'll take a look at your weight file and investigate further. Thanks for flagging this, and let's aim to resolve it soon! 🚀 — Reply to this email directly, view it on GitHub<#7928 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABJGLUENIVQIV3F5T3UN6LDYYLNJLAVCNFSM6AAAAABCS5GCYGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DQMBQHAYDK>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[glenn-jocher]

Hey Aslak! 🌟

Thanks for reaching out. Yes, aligning the image size for both training and prediction is crucial, and it's great that you've kept them consistent at 640. And yes, I've tested with the weight files on my end as well.

Since you're still experiencing issues, could you ensure that any preprocessing steps applied during training are also applied identically during prediction? Sometimes subtle differences in preprocessing can lead to discrepancies in performance.

If the issue persists, we might need to delve deeper into the model or the data. Feel free to share more details or snippets of your code if convenient!

Keep flying high with YOLOv8! 🚀

— Glenn

[Riyasharma-in]

I'm attempting to generate a heatmap across multiple streams in my instance. To achieve this, I've created multiple instances for each stream. However, I'm encountering a floating-point issue when fitting the heatmap. The heatmap generates floats at a time for a single stream, but I'm unsure how to address this problem across multiple streams. Any assistance on resolving this would be greatly appreciated.

[glenn-jocher]

Hi there! 👋 It sounds like you're making great progress with your instance and multiple streams! For addressing the floating-point issue when generating heatmaps across multiple streams, a common approach is to ensure that all your streams are synchronized in terms of processing rate and data type consistency.

A good starting point would be to explore the use of a consistent data format for all heatmaps before merging or combining them. For instance, you might want to standardize the floating-point precision across all streams, such as converting to the same float precision (e.g., float32 or float64) before fitting the heatmap.

Here's a very basic Python snippet demonstrating how you might convert an array to a consistent float type:

import numpy as np

# Example heatmap data for one stream
heatmap_data = np.array(your_heatmap_data, dtype=np.float32) # Convert to float32

By ensuring that all heatmap data across your streams is converted to the same data type, you might resolve inconsistencies when fitting the heatmap across multiple streams. If each stream's heatmap is generated in a slightly different format or precision, bringing them to a common ground before any further processing is essential.

Let me know if this helps, or if you're looking for something more specific! 😊

[Riyasharma-in]

I have a main function called heatmap_generate which returns an image after generating a heatmap. Now, I've created multiple streams for handling different sources. For instance, for camera one, I have a cam1_sender.py file, and for camera two, I have cam2_sender.py. Each of these files sends frames via a queue using different processes.

Before sending frames, I initialize the heatmap and send it through JSON. I run all the sender files simultaneously in parallel. Each sender file has its own receiver function.

The issue arises when multiple sender files are running simultaneously. Although they all call the same core logic function for generating the heatmap images, the initialization of the heatmap doesn't seem to work as expected. When I run a single sender file at a time, everything works fine. However, when multiple sender files are running, the generated heatmap images seem to lack the expected dynamics, appearing more like static circles rather than dynamically changing according to the movement of people.

I ensure that the heatmap is initialized when the first frame arrives, and then I globally reference that heatmap object. However, the problem persists.

Could you please provide some insights or assistance on how to resolve this issue?

[glenn-jocher]

@Riyasharma-in hey there! 👋 It sounds like you're experiencing a tricky situation with initializing and using your heatmap function across multiple streams. From what you've described, it seems that the global reference to the heatmap object might not be updating as expected across different processes.

Here's a quick suggestion: Instead of relying on a global object that might not be shared properly between processes, consider passing the heatmap state explicitly to each call of your heatmap generation function. This can ensure that each process works with the correct state without interference. For instance, you could modify your heatmap_generate function to accept the current state of the heatmap as an argument and return the updated state along with the image.

def heatmap_generate(frame, heatmap_state=None):
    # Your logic to update the heatmap based on the frame and current state
    updated_heatmap_state, image = ... # Processing logic
    return updated_heatmap_state, image

Then, in your sender files, make sure to manage the state of the heatmap for each frame processed.

This approach ensures each process maintains its own heatmap state, which could address the static circles issue by making the heatmap dynamics consistent with the movement observed in the frames.

Hope this helps and leads you toward a solution! If you have further questions or need more detailed assistance, feel free to ask. 😊

[arifan-instrumentasi-UB]

Hello, I am working with custom train yolo v8 model in Colab. I am using image as a source. How can I write a text on result image? I want to write the summary of prediction on the top of result image. Please help.

[glenn-jocher]

Hi Arifan,

Glad to help! 😊 To use the result of an inference directly without saving and reloading from a file path in OpenCV, you'll need to work directly with the image array returned by your YOLOv8 inference. After running an inference with your YOLO model, it typically returns an image tensor or array which you can convert to a format suitable for OpenCV operations. Here's a quick example:

from ultralytics import YOLO

# Assuming your model is loaded to 'model' and your image to infer is 'your_image'
model = YOLO('your_model.pt')
results = model('path/to/your_image.jpg')  # Perform inference

# Convert inference result to an image array format suitable for OpenCV
# Results.xyxy[0] to get detection bboxes, labels etc., depending on your model output
inference_img = results.imgs[0]

# Now you can use cv2.putText directly on the 'inference_img'
cv2.putText(inference_img, summary_text, position, font, font_scale, color, thickness)

# Show or Save 'inference_img' as needed
cv2.imshow('Inference Result with Text', inference_img)
cv2.waitKey(0)
cv2.destroyAllWindows()

In this example, results.imgs[0] gets the image array of the first (or only) image predicted by the model, which you can then manipulate using OpenCV functions like cv2.putText directly without needing to save and reload the image from a path.

Hope this clarifies things for you! Let me know if you have any more questions.

[arifan-instrumentasi-UB]

Hi, so helpful for me. thank you.

now, iam trying the same work using video file as an input. can i using the same code?

[glenn-jocher]

Hi there! 👋 Glad to hear the previous advice was helpful for you.

For processing a video file, the approach will be slightly different from handling a single image since you'll be working with a stream of frames rather than a static picture. You'll want to loop over each frame in the video, apply the detection, and then optionally write the text on each frame before displaying or saving it.

Here's a simplified snippet to get you started:

import cv2
from ultralytics import YOLO

# Load your custom-trained model
model = YOLO('path/to/your_model.pt')

# Initialize video capture from a file
cap = cv2.VideoCapture('path/to/your_video.mp4')

while cap.isOpened():
    ret, frame = cap.read()
    if not ret:
        break

    # Run model inference on the current frame
    results = model(frame)

    # [Optional] Customize this part to add your summary text to each frame
    cv2.putText(frame, 'Your summary text here', (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 0, 0), 2)

    # Display the frame
    cv2.imshow('Frame', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'): # Press 'q' to quit
        break

cap.release()
cv2.destroyAllWindows()

Make sure to adjust the paths, model, and summary text as needed. This code assumes you're working within an environment that supports OpenCV displays (like Jupyter or running locally).

For real-time performance or handling larger videos, consider optimizations 🚀 such as reducing imgsz for faster processing or using a more powerful GPU.

[arifan-instrumentasi-UB]

so we only need to put text on frame from cap.read() and no need to put on inference result?

[glenn-jocher]

Hi there! 😊

Yes, you're exactly right! For adding text to your video frames, you'll directly work with the frame obtained from cap.read() rather than the inference result. Here's how you can do it:

# Example code snippet
import cv2
cap = cv2.VideoCapture('path/to/your_video.mp4')

while True:
    ret, frame = cap.read()
    if not ret:
        break

    # Place your text on the frame here
    cv2.putText(frame, "Your Text Here", (10,30), cv2.FONT_HERSHEY_SIMPLEX, 1, (255,255,255), 2)

    # Show the frame
    cv2.imshow('Frame', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

This approach lets you dynamically display information directly on the video stream. 📹 If you have more questions or need further assistance, feel free to ask. Happy coding!

[ANAS1997AA]

How can I impose my own optimizer and learning rate to my yolov8OBB script. it is always set to auto and the LR is chosen automatically.

[ANAS1997AA]

thank you so much for your response. I want tp apply ReduceLROnPlateau and I encountered problems. Can you help me out ?

[ANAS1997AA]

I only have data.yaml . I import my dataset from roboflow in googlecolab and it contains only data.yaml

[glenn-jocher]

Hi there! 😊 It sounds like you're working directly with a data.yaml file from Roboflow which is great for importing your dataset into a Google Colab environment. If data.yaml is all you have, that's actually sufficient to get started with training using Ultralytics YOLOv8.

Here's a simple way to use it for training your model:

from ultralytics import YOLO

# Load YOLOv8 model, here we use a default model just for the example
model = YOLO('yolov8n.pt')

# Start training with your dataset
model.train(data='path/to/your/data.yaml')

Make sure you adjust the path to where your data.yaml is located in Colab. Since you're importing from Roboflow, your YAML file should already include correct paths to your images and labels alongside class names and counts, assuming the dataset was correctly set up in Roboflow.

For applying ReduceLROnPlateau, you might need to customize the training loop, which can be complex within Ultralytics YOLO's current setup as it involves deep integration with their training script. YOLOv8's architecture is highly optimized, and custom learning rate schedulers aren't directly supported through their high-level API or command line yet.

However, you can manually adjust learning rates between epochs by monitoring your validation performance, although automating this right now is a bit challenging without adjusting the source code.

I hope this helps! If you need further assistance, I highly recommend posting a detailed issue in the Ultralytics GitHub repository, where you can get more specific guidance tailored to your use case. 🚀

[ANAS1997AA]

thank you so much. can you please tell me how can I access to the code source and try to modify it ?

[glenn-jocher]

@ANAS1997AA hey there! 😄 We're thrilled to hear that you're interested in diving deeper into the source code of YOLOv8 and perhaps even tweaking it to fit your needs better. All of our code is hosted on GitHub, providing an open door for enthusiasts like you to explore, understand, and modify as you wish.

To get started, simply clone our repository:

git clone https://github.com/ultralytics/yolov8.git

After cloning, feel free to navigate through the codebase. If you're planning to modify the training process, you might want to check out scripts like train.py for entry points and models for model architectures.

Remember, after making your changes, always test to ensure everything runs smoothly. And, if you come up with improvements or fixes that could benefit others, consider creating a pull request! We love seeing community contributions. 🚀

Happy coding!

[kanosei]

Anyone seeing any difference when using apple gpu

with the following I get no validation prediction
yolo train data=/Users/user1/code/yolov8new/datasets/data.yaml device=mps

without it i do although its much slow.
yolo train data=/Users/user1/code/yolov8new/datasets/data.yaml

With same test data,

sample output of an epoc with mps. (notice some values not populated such as cls_loss and dfl_loss

  Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
  1/100         0G          0        nan          0         15        640: 100%|██████████| 21/21 [00:45<00:00,  2.16s/it]
             Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 1/1 [00:02<00:00,  2.52s/it]
               all         18         18          0          0          0          0

sample output without just using cpu

  Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
  1/100         0G     0.7308      2.364      1.143         15        640: 100%|██████████| 21/21 [02:15<00:00,  6.45s/it]
             Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 1/1 [00:04<00:00,  4.42s/it]
               all         18         18    0.00329          1      0.526      0.439

have verified that MPS is available with the following script

python metalTest.py
tensor([1.], device='mps:0')

cat metalTest.py
import torch
if torch.backends.mps.is_available():
mps_device = torch.device("mps")
x = torch.ones(1, device=mps_device)
print (x)
else:
print ("MPS device not found.")%

many thanks.

[glenn-jocher]

Hey there! It looks like you're encountering some issues with validation predictions when using the MPS device for training with YOLOv8. Your observation regarding the nan and 0 values in the cls_loss and dfl_loss on the MPS device as opposed to the CPU is indeed peculiar. 🤔

Given that your setup correctly recognizes the MPS device, the issue might not be with device availability but rather with how MPS interacts with the loss calculation or backpropagation in this specific scenario.

A few things to consider:

• PyTorch Version: Ensure you're using a version of PyTorch that fully supports MPS. PyTorch introduced MPS support in version 1.9, with subsequent updates providing improvements.
• Model Compatibility: Check if the model architecture has any compatibility issues with MPS or requires specific configurations when running on Apple's GPU.
• Loss Calculation: Investigate if certain operations involved in cls_loss and dfl_loss computation are behaving differently on MPS.

As a diagnostic step, you might want to try running a simpler model or a different task (if feasible) on the MPS to see if the issue persists. It could help isolate whether the problem is specific to the model or more widespread.

Here's an example command with added logging for more detailed insight during training (assuming your setup allows for verbose logging):

yolo train data=/Users/user1/code/yolov8new/datasets/data.yaml device=mps --verbose

If the problem remains, this could be a valuable point of feedback for the PyTorch community, given the relatively recent support for MPS.

Please keep us posted on your findings or if there's anything else we can help with. Your exploration helps improve the experience for everyone in the community! 🚀

[kanosei]

Many thanks for the response. Using version 2.2.2 of pytorch and can see GPU usage going up in activity monitor while running with the device=mps flag set.

python pytorchVersion.py
2.2.2

I did pull the sample coco yaml (https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/coco8.yaml) and ran on the CPU and got predictions made successfully. This is the output from the CPU run and i got a successful validation image generated.

Validating runs/detect/train2/weights/best.pt...
Ultralytics YOLOv8.2.2 🚀 Python-3.11.7 torch-2.2.2 CPU (Apple M1 Max)
Model summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 1/1 [00:00<00:00, 3.13it/s]
all 4 17 0.599 0.898 0.888 0.618
person 4 10 0.647 0.5 0.521 0.29
dog 4 1 0.315 1 0.995 0.597
horse 4 2 0.689 1 0.995 0.598
elephant 4 2 0.629 0.886 0.828 0.332
umbrella 4 1 0.539 1 0.995 0.995
potted plant 4 1 0.774 1 0.995 0.895
Speed: 1.6ms preprocess, 74.2ms inference, 0.0ms loss, 0.9ms postprocess per image
Results saved to runs/detect/train2

With the device=mps set i got the following response which looks similar but this time round the val predict image had no annotations.

Ultralytics YOLOv8.2.2 🚀 Python-3.11.7 torch-2.2.2 MPS (Apple M1 Max)
Model summary (fused): 168 layers, 3151904 parameters, 0 gradients, 8.7 GFLOPs
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 1/1 [00:00<00:00, 1.56it/s]
all 4 17 0.553 0.719 0.755 0.529
person 4 10 0.664 0.5 0.519 0.278
dog 4 1 0 0 0.199 0.0752
horse 4 2 0.716 1 0.995 0.598
elephant 4 2 0.605 0.816 0.828 0.332
umbrella 4 1 0.55 1 0.995 0.995
potted plant 4 1 0.78 1 0.995 0.895
Speed: 1.4ms preprocess, 138.6ms inference, 0.0ms loss, 1.4ms postprocess per image
Results saved to runs/detect/train3

From the output it looks like the run is successfully identifying the objects but the model doesn't work and the validation image isnt updated. Though initially its my model size but that cant be the case based on the text being output.

Anything you notice in what im doing wrong to give different behaviour.

[glenn-jocher]

Hey! It's great to see you're exploring YOLOv8 with MPS on your M1 Max. 👍 From your description, it looks like the training and validation are behaving differently when using device=mps compared to when running on the CPU. Given that the GPU usage is up (which is a good sign that MPS is engaged), but validation images lack annotations, there might be nuances with MPS handling or model compatibility.

Since you're already on PyTorch 2.2.2, compatibility with MPS should generally be good. However, it's worth checking a few points:

• Ensure the latest version of Ultralytics YOLO to grab any recent fixes or improvements specifically for MPS.
• Double-check the model output details in both scenarios (CPU vs. MPS) to spot any discrepancies early in the process.

Though it's less common, there might be edge cases where MPS behaves slightly differently, affecting the final output. Since the training outputs look okay but the validation annotated images are empty, it's possible that something specifically affects how annotations are applied or saved.

As a workaround, you might try running predictions on individual images using the model saved from MPS training to see if the issue persists across different operations.

yolo detect predict model=path/to/mps/trained/model.pt source='path/to/test/image.jpg'

If the issue remains, it could be beneficial to raise this as a discussion in the Ultralytics GitHub or forums, as it might require a deeper dive into the model's MPS compatibility or reveal a current limitation. The community or the contributors might also have insights or workarounds for this.

Keep experimenting, and don't hesitate to share your findings or further questions! 🚀

[kanosei]

Thanks again for your help.

I set up a clean python venv and tried again so should be running latest ultralytics and see the same issue. I have run the model on CPU locally with success and also on Colab with a TPU with success so to me is definitely something to do with the MPS gpu.

I also tried calling the yolo detect predict as suggested and don't get an annotated images. Will raise the issue on github as suggested.

many thanks

[glenn-jocher]

Hey there! 👋

It sounds like you've done quite a bit of troubleshooting already – thanks for being so thorough! Since you've successfully run your model on CPU and TPU but are facing issues with MPS, it does seem to hint towards an MPS-specific quirk.

Clean Python environments and checking against the latest Ultralytics version are great steps. Given the inconsistencies with annotations not appearing in output images when leveraging MPS, raising a GitHub issue is the right move. This way, the community and developers can dive deeper into potential MPS-related nuances or bugs.

In the meantime, if you're experimenting further, you might want to try using different versions of PyTorch that have known good support for MPS, just in case there's a version-specific behavior affecting YOLOv8's output.

Best of luck, and we're looking forward to seeing your issue on GitHub to help get this sorted! 🚀

Cheers!

[arifan-instrumentasi-UB]

Hello, I have PCB that include 1 LED, 1 red wire, and 1 black wire. I want to detect reject/good LED and do color recognition on wire using open cv. Is it possible to do both at the same time and show the bounding box (put object detection bounding box and color recognition bounding box at the same frame). please help. maybe I need example code. Thank You.

[pderrenger]

Hello! Yes, it's absolutely possible to perform object detection and color recognition simultaneously using YOLO for object detection (detecting the LED and wires) and OpenCV for color recognition within the detected bounding boxes. 😊

Here’s a simplified example to guide you:

1. Object Detection with YOLO: First, detect objects with YOLO and get the bounding boxes.

from ultralytics import YOLO
import cv2

# Load your trained model
model = YOLO('path/to/your/model.pt')

# Process an image
image = cv2.imread('path/to/your/image.jpg')
results = model(image)

# Extract bounding boxes
bboxes = results.boxes.xyxy[0]  # Format: [x1, y1, x2, y2, conf, cls]

2. Color Recognition with OpenCV: For each bounding box, use OpenCV to recognize the color within that region.

for bbox in bboxes:
    x1, y1, x2, y2 = map(int, bbox[:4])

    # Crop to bounding box
    crop_img = image[y1:y2, x1:x2]

    # Convert to HSV and apply color recognition logic here
    # E.g., using cv2.inRange() for specific color ranges

3. Display Results: Finally, display or save the image with bounding boxes and identified colors.

    # Draw bounding box and text for the recognized color
    cv2.rectangle(image, (x1, y1), (x2, y2), (0, 255, 0), 2)
    cv2.putText(image, 'Color Name', (x1, y1-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (36,255,12), 2)

cv2.imshow('Result', image)
cv2.waitKey(0)

Note: For color recognition, you'll need to define the HSV color ranges that correspond to your desired colors (e.g., red and black wires). Each bounding box area can be individually processed to determine its dominant color.

I hope this helps! Feel free to ask if you have any more questions. 🌟

[Githubprojectsacc]

Hello! I want to get the metric values like map score for object detection for hyper parameter tuning using yolov8 model. I used the same code the above for finetuning with ASHA and hyperopt search algorithm. In the training function I returned the evaluation score .But in the result I got the same metric values in every trial .Can u pls help me with this.

[glenn-jocher]

Hello! 😊 It sounds like you're facing an issue with obtaining different metric values across trials during hyperparameter tuning with the YOLOv8 model. If you're getting the same metric values in every trial, it might suggest the model isn't being re-initialized or properly updated between trials.

To address this, please ensure that for each trial, the model is indeed being re-loaded or re-initialized with the new set of hyperparameters. Also, confirm that the evaluation dataset being used does not remain constant (unless intended) and that data augmentation or any other experimental variances are applied correctly.

Here's a small check you can include before each training run to ensure your model and data are set up correctly for each hyperparameter tuning trial:

# Pseudo-code - Adjust according to your actual setup
for trial in hyperopt_trials:
    model = YOLO('path/to/model/config', override=trial.params)  # Load model with new hyperparameters
    results = model.train('path/to/custom_dataset.yaml')
    eval_score = model.val('path/to/val_dataset.yaml')  # Evaluate model
    trial.report(eval_score, step=0)

Ensure that model, results, and eval_score are correctly reassigned for each trial iteration. Also, make sure the training and validation functions are properly capturing the dynamic changes in hyperparameters.

Hope this helps! If the issue persists, providing more specific code snippets might help diagnose the problem further. Don't hesitate to reach out if you have more questions.

[PiaMoe]

Hello,
I want to continue a training of a yolov8 model with some new data. When I previously trained it, I used two graphic cards but right now I have only one available. So I load my last.pt model, set resume=True and device=0. But it still expects two devices, as I get this error: ValueError: Invalid CUDA 'device=0,1' requested. Use 'device=cpu' or pass valid CUDA device(s) if available, i.e. 'device=0' or 'device=0,1,2,3' for Multi-GPU.

If I set resume=False the training starts with my current settings. So is this error occuring because I previously trained with two devices? If yes, can I somehow change this?

[pderrenger]

Hello! 😊

Yes, the issue you're encountering is due to the original training being done on two GPUs, and now you're attempting to resume on a single GPU. A quick and effective solution is to modify the .pt file explicitly to adapt to your current GPU setup.

Here's how you can adjust your model to resume training on a single GPU:

1. Load your last.pt model in a Python environment.
2. Update the model’s device settings to match your current hardware.
3. Save the modified model and resume training.

Here's a code snippet to guide you:

import torch

# Load the model
model = torch.load('path/to/your/last.pt')

# Modify the device setting
model['device'] = '0'  # Update to your current device setup

# Save the modified model
torch.save(model, 'path/to/your/modified_last.pt')

# Now you can resume training with the modified model

After this, try resuming training with resume=True and device=0 using the modified_last.pt model. This should align the model's device setup with your current hardware configuration.

Hope this helps! Let us know if you have any other questions. 🚀

[Amu1620]

hi all i want to ask regading supervision
in supervisison,linecounter i used IN and OUT condition for object count but i stuck in to find the duplicate count means at same position IN count+1 and OUT at same position +1 when i want to find out duplication form In and out in position wise is it possible

[glenn-jocher]

@Amu1620 hello! It sounds like you're trying to manage the count of objects entering and exiting at the same position and want to avoid duplicating counts. A good approach would be to track the IDs of the objects as they move in and out.

You might consider implementing a tracking algorithm like Deep SORT which can provide unique IDs for each detected object. This way, you can increment your IN and OUT counters based on these unique IDs, ensuring that each object is counted only once when it crosses a defined line or area.

Here's a simplified example of how you might adjust your existing setup to use object IDs for counting:

# Assuming 'trackers' is a list of objects being tracked with unique IDs
for tracker in trackers:
    if tracker.crosses_line('in') and not tracker.counted:
        in_counter += 1
        tracker.counted = True
    elif tracker.crosses_line('out') and not tracker.counted:
        out_counter += 1
        tracker.counted = True

In this pseudo-code, crosses_line is a method you'd define based on your line-crossing logic, and counted is a flag to check if an object has already been counted in a specific direction.

This method should help you manage IN and OUT counts without duplication. If you need further assistance, don't hesitate to ask! 😊

[Amu1620]

Thank you so much for your response,I will use this method.

[glenn-jocher]

@Amu1620 you're welcome! If you need further assistance or have more questions, feel free to reach out. Happy coding! 😊 If you need more detailed guidance, don't forget to check out our FAQ section for common issues and solutions.

[Amu1620]

Thank you for your response. I have one dout I detect and count the objects vertically but not able to count horizontal count. Only single object for both vertical and horizontal. Here in this code in and out count vertically but not able to count horizontally. Can you please share a code real time object detection tracking and counting and find duplication of object I used video file video is up and down,tracking, counting but stuck in duplication,And horizontal counting share the solution. Regards, Amruta patil

…

On Sat, 4 May, 2024, 11:15 pm Glenn Jocher, ***@***.***> wrote: @Amu1620 <https://github.com/Amu1620> you're welcome! If you need further assistance or have more questions, feel free to reach out. Happy coding! 😊 If you need more detailed guidance, don't forget to check out our FAQ section <https://docs.ultralytics.com/help/FAQ/> for common issues and solutions. — Reply to this email directly, view it on GitHub <#7928 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BANL2BTRMET7GGEZR3LD6Y3ZAUNCTAVCNFSM6AAAAABCS5GCYGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMJVGQZTI> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[glenn-jocher]

Hello Amruta,

For counting objects horizontally in addition to vertically, you'll need to modify your approach based on the direction of object movement and area counting. You may use different tracking IDs assigned to each unique object to handle duplication concerns effectively.

Here’s a basic example to adjust tracking for both horizontal and vertical movements:

from ultralytics import YOLO

# Load a pretrained model
model = YOLO('yolov8n.pt')

# Detect objects in a video
results = model.track('path/to/your/video.mp4', stream=True)

horizontal_count = 0
vertical_count = 0
track_ids = set()

for result in results:
    for *box, conf, cls, track_id in result.boxes:
        if track_id not in track_ids:
            if box[0] > box[2]:  # Horizontal movement
                horizontal_count += 1
            else:  # Vertical movement
                vertical_count += 1
            track_ids.add(track_id)

print(f'Horizontal count: {horizontal_count}, Vertical count: {vertical_count}')

This code checks the orientation of detected bounding boxes to determine the direction of object movement and increments the respective counters. The track_id helps in recognizing new and existing objects, avoiding duplicates in your counts.

If you need further customization, you might need to adjust detection thresholds, area of interest (AOI) settings, or use more advanced tracking algorithms like Deep SORT for better handling of occlusions and object interactions.

Don't hesitate to revisit the FAQ or check out our more detailed guides on object tracking.

[Amu1620]

Thank you so much Glenn for your provided logic.I implement this logic.I need one more logic can I apply stitching or any other method to calculate count of object per wall.I working on room data and i want to find out different object at each wall and count of that objects also.can you give me any logic or method

…

On Mon, May 6, 2024 at 12:19 AM Glenn Jocher ***@***.***> wrote: Hello Amruta, For counting objects horizontally in addition to vertically, you'll need to modify your approach based on the direction of object movement and area counting. You may use different tracking IDs assigned to each unique object to handle duplication concerns effectively. Here’s a basic example to adjust tracking for both horizontal and vertical movements: from ultralytics import YOLO # Load a pretrained modelmodel = YOLO('yolov8n.pt') # Detect objects in a videoresults = model.track('path/to/your/video.mp4', stream=True) horizontal_count = 0vertical_count = 0track_ids = set() for result in results: for *box, conf, cls, track_id in result.boxes: if track_id not in track_ids: if box[0] > box[2]: # Horizontal movement horizontal_count += 1 else: # Vertical movement vertical_count += 1 track_ids.add(track_id) print(f'Horizontal count: {horizontal_count}, Vertical count: {vertical_count}') This code checks the orientation of detected bounding boxes to determine the direction of object movement and increments the respective counters. The track_id helps in recognizing new and existing objects, avoiding duplicates in your counts. If you need further customization, you might need to adjust detection thresholds, area of interest (AOI) settings, or use more advanced tracking algorithms like Deep SORT for better handling of occlusions and object interactions. Don't hesitate to revisit the FAQ <https://docs.ultralytics.com/help/FAQ/> or check out our more detailed guides on object tracking. — Reply to this email directly, view it on GitHub <#7928 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BANL2BQYNWT5MHVDXKFSXZTZAZ5K7AVCNFSM6AAAAABCS5GCYGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMRRGU4DS> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[glenn-jocher]

Hi there,

For counting and distinguishing objects on each wall in your room data, you could consider using object detection in combination with perspective transforms or image stitching if you have multiple camera angles. This can help you accurately identify and count objects across different walls.

Here's a broad idea:

1. Use YOLOv8 to detect objects in each room image.
2. Employ image stitching techniques to combine images from different angles into a single panoramic view, or use perspective transforms to align and scale images according to the room's geometry.
3. Count and classify detected objects in stitched or transformed images, ensuring each object is assigned to its respective wall.

Here's a snippet using YOLOv8 to detect objects:

from ultralytics import YOLO

# Load the YOLOv8 model
model = YOLO('yolov8n.pt')

# Detect objects in an image
results = model('your_room_image.jpg')

# Print detected objects and their counts
print(results.pandas().xyxy[0])

For image stitching or perspective transforms, you might need to look into OpenCV functions like cv2.warpPerspective() for manual alignment or cv2.createStitcher() for automated stitching.

Let me know if this helps or if you need further assistance!

Best regards.

[kamrandar]

how can i hand the problem where i have press machine and i have actually see if it i running and how many presses it did with in particular time-feed on in a video

[glenn-jocher]

To monitor a press machine's operation and count the number of presses within a specific time frame using a video feed, you can utilize Ultralytics YOLO for object detection. Here's a brief overview of how you could set this up:

1. Prepare your dataset: You'll need to gather video footage of the press machine in operation. Annotate these videos to mark each press as it occurs.

2. Train a custom YOLO model: Fine-tune a pre-trained YOLO model on your annotated dataset. Use a dataset configuration file (YAML) that points to your annotated frames and specifies the number of classes (in this case, just the press).

3. Perform real-time detection: Once the model is trained, run it on your video feed to detect the press operation in real-time. You can set up a counter within your detection script to count the presses detected in specified time intervals.

Here's a simple Python code snippet to give you an idea:

from ultralytics import YOLO

# Load the trained model
model = YOLO('path/to/your/trained/model.pt')

# Real-time detection with video source
video_path = 'path/to/your/video.mp4'
results = model.track(source=video_path, show=True)

press_count = 0
for frame_results in results:
    for det in frame_results.boxes:
        if int(det.cls) == PRESS_CLASS_ID:  # Assuming 'PRESS_CLASS_ID' is the relevant class ID for press operation
            press_count += 1

print(f"Total presses detected: {press_count}")

Modify paths and class ID as per your setup. If you need further assistance or customization, don't hesitate to explore more in the Ultralytics documentation. 🚀

[felipequentino]

I'm trying to build a model to detect knifes and handguns (pistols and revolvers), the training batch is like 1000 images (i know it's little), and a question that i have is: get a picture of a handgun in a desk, showing her entirely, and get another pic of a man holding a handgun, showing only the upper part of the pistol, and some times the trigger. I make another class for this situation (only the upper part), or I leave this way?

[pderrenger]

Hello! It's great to see you're working on a model for detecting specific objects like handguns and knives. When training your detection model with the YOLOv8, considering your situation where the object appears partially in some images (like a handgun held by a person), you don't necessarily need to create a separate class for the partial visibility of the handgun.

Instead, keep it as a single class. YOLO models are quite robust and can learn to detect objects even from partial views, given enough representative training data. Make sure to include various images showing different parts and orientations of the handguns and knives in your training set. Data augmentation techniques might also help the model generalize better from your limited dataset.

Here’s a simple example of how you might handle your dataset configuration in YAML:

# Example of dataset configuration for custom training
train: /path/to/your/train/images
val: /path/to/your/validation/images

# Classes
nc: 2  # number of classes
names: ['knife', 'handgun']

Ensure that your images are well-annotated, highlighting all situations—both full and partial visibility. The model's performance largely depends on the variety and quality of the training data.

Best of luck with your model training! If you encounter any more issues or have questions, feel free to reach out. 😊

[felipequentino]

Thanks!
In handgun case, I split this class in two: pistols and revolvers, because I thought that are two different objects (even though they are two objects that belong to the same class: handgun). But in the knife question, I had to split in 4 classes: a Kitchen knife, a combat knife, a pocket knife and even a cleaver. I did this because of the same reason of the handguns.

This was necessary, or am I going to have more work this way?

[pderrenger]

@felipequentino hello! 🌟 It sounds like you've tailored your class design to capture distinct characteristics of handguns and knives, which can be quite beneficial for specific applications where such granularity is necessary.

Splitting the handgun class into pistols and revolvers, and the knife class into multiple types based on their distinct features, such as kitchen knives and combat knives, makes sense if your model's application requires accurately distinguishing among these subtypes.

This approach might indeed involve more work initially in terms of gathering and labeling a diverse dataset for each subtype. However, if distinguishing between these is crucial for your use case, the additional effort could significantly improve your model's utility and accuracy.

If you ever find the distinctions don't add value to the problem you're solving, you could consider merging some classes to streamline the model without compromising performance. Best of luck with your project! If you need further assistance down the line, feel free to reach out. 😊

[Mrgorlay]

Hello, could you give me some help and advice? I hope I'm not bothering you too much.
I have a question. I'm using the YOLOv8 model in .pt format for inference in a web application built with the Django framework. I've purchased an A16 graphics card with 8 GPUs. Is there a good way to utilize all GPUs without specifying the device? Currently, I'm randomly selecting a device using the following code:
device_num = random.randint(0, 7)
results = modelobj(source=img, device=device_num, half=True)
I'd appreciate any guidance on how to make the API more universally applicable.
@glenn-jocher thanks!

[Mrgorlay]

hello,@glenn-jocher ,
Thank you very much for your enthusiastic response.

def infer(img):
    # img 已经是一个numpy数组（BGR格式）
    # image = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    image = Image.fromarray(img)
    image_pil = image.resize((640, 640))
    image = np.array(image_pil).astype(np.float32) / 255.0
    image = np.transpose(image, (2, 0, 1))  # 转换为形状 (3, 640, 640)
    image = np.expand_dims(image, axis=0)  # 增加批次维度, 可以同时传入多个，需要把N设置，形状变为 (1, 3, 640, 640)
    ort_inputs = {sess.get_inputs()[0].name: image}
    ort_outs = sess.run(None, ort_inputs)
    

If I understand correctly, here are a few ideas I'd like to discuss:
(1) I'd like to extract names from the metadata of the ONNX model, then get the names of 'xywhconf' and 'id' from ort_outs, and combine them with names. This way, I can return 'xywhconf' and 'name' to the client using a response.
(2) The second idea is to draw the position, confidence, and type of defects on the image and return it.
(3)If there are multiple defect types in ort_outs, how can I control the status (OK or NG) of defects by setting the confidence of multiple defect types?

Could you please explain the implementation methods? I'm not very familiar with ONNX and still in the exploration stage. I understand how to do inference with PyTorch. Thank you."

[glenn-jocher]

Hello @Mrgorlay,

Thank you for your detailed inquiry and for exploring ONNX with YOLOv8! Here are some insights and suggestions for your questions:

1. Extracting Names and Combining with Outputs:

   • You can extract output names directly from the ONNX model using sess.get_outputs(). Each output object has a .name attribute that you can match with your desired outputs like 'xywhconf' and 'id'.
   • Once you have the names and corresponding outputs, you can format them as needed to send back to the client.

2. Drawing on Images:

   • To draw the detection results (position, confidence, and type) on the image, you can use OpenCV. Here’s a basic example:

     import cv2
     image = cv2.imread('path_to_image.jpg')
     for detection in ort_outs:
         x, y, w, h, conf, cls_id = detection
         cv2.rectangle(image, (x, y), (x + w, y + h), (255, 0, 0), 2)
         cv2.putText(image, f'{cls_id} {conf:.2f}', (x, y-10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (255,0,0), 2)
     cv2.imwrite('output.jpg', image)

3. Handling Multiple Defect Types:

   • To control the status (OK or NG) based on multiple defect types and their confidences, you can set a confidence threshold for each defect type. If any defect’s confidence is below its respective threshold, you could classify the status as NG, otherwise OK.
   • Implement this by defining a dictionary of thresholds for each defect type and checking each detection against its threshold.

Here's a simple example:

thresholds = {0: 0.5, 1: 0.6}  # class_id: threshold
status = "OK"
for detection in ort_outs:
    _, _, _, _, conf, cls_id = detection
    if conf < thresholds.get(cls_id, 1):
        status = "NG"
        break

These steps should help you implement the desired functionalities in your Django API using the ONNX model. If you have further questions or need more detailed guidance, feel free to ask. Happy coding! 😊

[glenn-jocher]

Hello @Mrgorlay,

Thank you for sharing your detailed code and explanation! It looks like you're making great progress with ONNX inference. Let's address your questions and potential issues:

1. Extracting Names and Combining with Outputs:
   To extract metadata and combine it with the inference results, you can use the following approach:

   # Extract metadata
   input_name = sess.get_inputs()[0].name
   output_name = sess.get_outputs()[0].name
   
   # Perform inference
   ort_inputs = {input_name: image}
   ort_outs = sess.run(None, ort_inputs)
   
   # Process outputs
   output = np.squeeze(ort_outs[0]).transpose()
   boxes = output[:, :4]
   scores = output[:, 4]
   classes = np.argmax(output[:, 5:], axis=1)
   
   # Combine results with class names
   class_names = ['class1', 'class2', 'class3']  # Replace with your actual class names
   results = []
   for i in range(len(boxes)):
       if scores[i] >= conf_thres:
           result = {
               'box': boxes[i].tolist(),
               'score': float(scores[i]),
               'class': class_names[classes[i]]
           }
           results.append(result)

2. Drawing on Images:
   To draw bounding boxes, confidence scores, and class names on the image:

   for i in range(len(boxes)):
       if scores[i] >= conf_thres:
           x, y, w, h = boxes[i]
           x1, y1 = int(x), int(y)
           x2, y2 = int(x + w), int(y + h)
           cls_id = classes[i]
           conf = float(scores[i])
           cv2.rectangle(img, (x1, y1), (x2, y2), (255, 0, 0), 2)
           cv2.putText(img, f'{class_names[cls_id]} {conf:.2f}', (x1, y1 - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.9, (255, 0, 0), 2)
   cv2.imwrite('output.jpg', img)

3. Handling Multiple Defect Types:
   To control the status based on multiple defect types and their confidences:

   thresholds = {0: 0.5, 1: 0.6}  # Example thresholds for each class
   status = "OK"
   for i in range(len(boxes)):
       if scores[i] >= thresholds.get(classes[i], 1):
           status = "NG"
           break

These steps should help you implement the desired functionalities in your Django API using the ONNX model. If you encounter any issues or need further assistance, please provide a minimum reproducible example so we can better understand and address the problem. You can find more details on creating a reproducible example here.

Keep up the great work, and feel free to reach out with any more questions! 😊

[Mrgorlay]

Hello, thank you for your guidance.@glenn-jocher ,

I've implemented basic functionalities. I'd like to ask you a question: How can I retrieve the names from metadata in YOLOv8's ONNX model?

[glenn-jocher]

Hello @Mrgorlay,

Thank you for your question and for implementing the basic functionalities! 😊

To retrieve the class names from the metadata in a YOLOv8 ONNX model, you can follow these steps:

1. Extract Metadata: ONNX models typically don't store class names directly. Instead, you can store the class names in a separate file or hard-code them in your script. If you have the class names stored in a file, you can load them as follows:

   # Assuming class names are stored in a file named 'classes.txt'
   with open('classes.txt', 'r') as f:
       class_names = f.read().splitlines()

2. Combine with Inference Results: After performing inference, you can map the class indices to their respective names:

   import numpy as np
   
   # Perform inference
   ort_inputs = {sess.get_inputs()[0].name: image}
   ort_outs = sess.run(None, ort_inputs)
   
   # Process outputs
   output = np.squeeze(ort_outs[0]).transpose()
   boxes = output[:, :4]
   scores = output[:, 4]
   classes = np.argmax(output[:, 5:], axis=1)
   
   # Combine results with class names
   results = []
   for i in range(len(boxes)):
       if scores[i] >= conf_thres:
           result = {
               'box': boxes[i].tolist(),
               'score': float(scores[i]),
               'class': class_names[classes[i]]
           }
           results.append(result)

3. Return or Display Results: You can then return these results in your API response or draw them on the image as needed.

If you encounter any issues or need further assistance, please ensure you are using the latest versions of torch and ultralytics. If the problem persists, providing a minimum reproducible example will help us investigate further. You can find more details on creating a reproducible example here.

Keep up the great work, and feel free to reach out with any more questions! 😊

[mayurkatre18]

from ultralytics import YOLO

model = YOLO("best.pt")
model.export(format='tflite')'''

Hi, I have to export model into tflite format but the error I'm getting is as given below.

TensorFlow SavedModel: export failure ❌ 19.1s: generic_type: cannot initialize type "StatusCode": an object with that name is already defined

ImportError Traceback (most recent call last)
in <cell line: 4>()
2
3 model = YOLO("/content/runs/segment/train/weights/best.pt")
----> 4 model.export(format='tflite')

11 frames
/usr/local/lib/python3.10/dist-packages/tensorflow_lite_support/python/task/vision/core/tensor_image.py in
17
18 from tensorflow_lite_support.python.task.vision.core import color_space_type
---> 19 from tensorflow_lite_support.python.task.vision.core.pybinds import image_utils
20
21

ImportError: generic_type: cannot initialize type "StatusCode": an object with that name is already defined.

Please give suggestion on this.

[pderrenger]

Hi there! It looks like you're encountering an ImportError related to TensorFlow Lite's Python bindings. This issue might be due to conflicting TensorFlow versions or duplicate installations.

Here's a quick suggestion to potentially resolve this issue:

1. Ensure that you have a clean environment by uninstalling and then reinstalling TensorFlow and TensorFlow Lite:

   pip uninstall tensorflow tensorflow-lite
   pip install tensorflow tensorflow-lite

2. After reinstalling, try exporting your model again:

   from ultralytics import YOLO
   
   model = YOLO("best.pt")
   model.export(format='tflite')

If the problem persists, it might be helpful to check for any known issues on the TensorFlow GitHub page or seek further assistance there.

[ping505]

Hello, I am using the YOLOv5 model to perform object detection on a pile of stacked parts. However, there have been instances where the recognition performance of individual parts is not good, as well as instances where the val/obj loss value increases and the val/cla loss first decreases and then increases. I don't know how to solve it, may I ask how to solve it? I really appreciate it

[glenn-jocher]

Hello! It sounds like you're experiencing some challenges with model performance and loss metrics. Here are a couple of suggestions that might help:

1. Data Augmentation: Enhance your dataset with more varied examples. This can help improve the model's ability to generalize and perform better on individual parts.

2. Hyperparameter Tuning: Adjust learning rates, batch sizes, or try different optimizers. Sometimes, fine-tuning these parameters can help stabilize the loss values.

3. Review Annotations: Ensure that your dataset annotations are accurate and consistent. Incorrect labels or bounding boxes can significantly affect performance.

4. Advanced Techniques: Consider using techniques like transfer learning from a similar task or applying more complex architectures if your hardware allows.

For more detailed guidance, you might find our FAQ section helpful, especially regarding fine-tuning models and improving accuracy. If the issue persists, feel free to share more details or reach out on our GitHub page! 🚀

[glenn-jocher]

Hello! It sounds like you're experiencing some challenges with model performance and training stability. Here are a couple of suggestions that might help:

1. Data Augmentation: Enhance your dataset with more varied examples. This can help improve the model's ability to generalize and perform better on individual parts recognition.

2. Hyperparameter Tuning: Adjust learning rates or try different optimizers. Sometimes, tweaking these can help stabilize training and improve loss metrics.

3. Model Complexity: Consider using a more complex version of the YOLO model if your hardware allows. This could potentially improve recognition accuracy.

4. Regularization Techniques: Implement techniques like dropout or batch normalization to prevent overfitting, which might be causing the increase in validation loss after initially decreasing.

For more detailed guidance, you might find the Ultralytics YOLO FAQ helpful. Keep experimenting, and good luck! 🚀

[ping505]

thank you

[glenn-jocher]

@ping505 you're welcome! If you have any more questions or need further assistance, don't hesitate to reach out. Happy detecting! 🚀

[snekumar]

how can I avoid Catostrophic forgetting in the fine tune YOLO model ?

[kamrandar]

you can use PEFT Techniques where most of the pre-trainned weights are left unchanged or use multi task fine tunning at the same time.

[glenn-jocher]

Thank you for sharing these techniques! Employing Progressive Elastic Fine-Tuning (PEFT) or multitask fine-tuning can indeed be effective strategies to mitigate catastrophic forgetting when fine-tuning YOLO models. Keeping most pretrained weights frozen ensures that the model retains knowledge from the original dataset while adapting to new data. Multitask learning, where the model learns from multiple tasks simultaneously, can also help in stabilizing the learning process across different datasets. These methods contribute to a more robust model by preserving previously learned features. 🚀

[YeapChweap]

Hello! I'm using yolov8 and I just can't use it from CLI.

I have installed ultralytics as below

ultralytics.checks()
Ultralytics YOLOv8.2.28 🚀 Python-3.11.9 torch-2.2.1+cu118 CUDA:0 (NVIDIA GeForce RTX 3060 Laptop GPU, 6144MiB)
Setup complete ✅ (16 CPUs, 15.3 GB RAM, 210.1/237.3 GB disk)

but everytime when I want to use the yolo command, it appears like this.

'yolo' is not recognized as an internal or external command operable program or batch file

[pderrenger]

@YeapChweap hello! 👋

Thank you for reaching out. It looks like you're encountering an issue with the yolo command not being recognized in your command line interface. Let's troubleshoot this together.

Steps to Resolve:

1. Ensure Installation Path is Correct:

   • Sometimes, the yolo command might not be recognized if the installation path is not added to your system's PATH environment variable. You can check if the ultralytics package is installed correctly by running:

     pip show ultralytics

   • If it is installed, ensure that the directory containing the yolo executable is in your system's PATH. You can add it manually if necessary.

2. Reinstall Ultralytics:

   • Try reinstalling the ultralytics package to ensure that all components are correctly installed:

     pip uninstall ultralytics
     pip install ultralytics

3. Check Python Scripts Directory:

   • Ensure that the directory where Python scripts are installed is included in your PATH. For example, on Windows, this might be something like C:\Users\<YourUsername>\AppData\Local\Programs\Python\Python<version>\Scripts.

4. Verify Python and Ultralytics Versions:

   • Make sure you are using the latest versions of torch and ultralytics. You can upgrade them using:

     pip install --upgrade torch ultralytics

Example of Adding to PATH (Windows):

1. Open the Start Search, type in "env", and select "Edit the system environment variables".
2. In the System Properties window, click on the "Environment Variables" button.
3. In the Environment Variables window, find the PATH variable in the "System variables" section, and click "Edit".
4. Click "New" and add the path to the directory where the yolo executable is located, then click "OK".

Example of Adding to PATH (Linux/Mac):

Add the following line to your .bashrc or .zshrc file:

export PATH="$PATH:/path/to/python/scripts"

Then, reload the shell configuration:

source ~/.bashrc  # or source ~/.zshrc

Additional Resources:

For more detailed troubleshooting steps, you can refer to our FAQ section.

If the issue persists, please provide more details or a minimum reproducible example so we can further investigate. You can find guidelines on creating a minimum reproducible example here.

I hope this helps! Feel free to reach out if you have any more questions. 😊

[YeapChweap]

Sorry but all of those did not work properly. But I have reinstalled ultralytics, the command prompt showed as belows

Installing collected packages: ultralytics
WARNING: The scripts ultralytics.exe and yolo.exe are installed in 'C:\Users\HP\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

What can I do now? I'm using Window 11 and my python version is 3.11.9. Thanks a lot for your reply.

[pderrenger]

Hello @YeapChweap,

Thank you for your patience and for providing the additional details. It looks like the issue is related to the yolo command not being recognized due to the installation path not being included in your system's PATH environment variable.

Steps to Resolve:

1. Add the Scripts Directory to PATH:

   • The warning message indicates that the scripts are installed in C:\Users\HP\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts. You need to add this directory to your system's PATH.

2. How to Add to PATH (Windows 11):

   • Open the Start menu and search for "Environment Variables".
   • Click on "Edit the system environment variables".
   • In the System Properties window, click on the "Environment Variables" button.
   • In the Environment Variables window, find the PATH variable in the "System variables" section, and click "Edit".
   • Click "New" and add the path C:\Users\HP\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\Scripts.
   • Click "OK" to close all windows.

3. Verify the PATH Update:

   • Open a new Command Prompt window and type:

     echo %PATH%

   • Ensure that the path you added appears in the output.

4. Test the yolo Command:

   • Close and reopen your Command Prompt to apply the changes.
   • Run the yolo command again to see if it is now recognized:

     yolo --version

Additional Resources:

For more detailed troubleshooting steps, you can refer to our FAQ section.

If the issue persists, please provide any error messages or additional details so we can further assist you. We're here to help! 😊