[Feat]: Utilize multiple GPUs and increase thread count during image captioning

[jferments]

Describe your use-case.

I recently was using OneTrainer UI to auto-caption ~750k images using WD14 tagger. I have a dual GPU machine, but it was only utilizing one of them (and was running at very low VRAM/power usage). Additionally, it was barely utilizing CPU at all.

I was wondering if there is any way to make it more fully utilize CPU/GPU resources to speed up batch processing of images. I have a 3.7 million image dataset I need to do next, and it could cut many hours off my processing time to be able to use multiple GPUs.

What would you like to see as a solution?

An option to select multiple GPUs to use, as well as increasing thread count for captioning.

Have you considered alternatives? List them here.

No response

[O-J1]

No support. Most developers for OT do no think its worth it and ruled it out multiple times. PR's are welcome.

[jferments]

Do you have links to tickets where they've rejected this idea? I didn't see any for it when I searched

[yggdrasil75]

#69 though I fully agree, using my 3080 and 3090 together would definitely be beneficial. especially when I want to train at 2048x2048 with better parameters.

[Disty0]

You can run separate taggers at the same time. You will start to get CPU bottlenecked to 60~ images per second per tagger.

Run one part of the dataset on GPU0 and other on GPU1.
And do not run anything on the CPU since your CPU will get hit pretty hard with all of those image operations.