How to utilize multi-core CPUs and GPUs

[junyixia]

I tried to run the sample code in the quick start section on a new computer with 64 cores CPU and 4 RTX A6000 GPU cards. However, I observed that only 4 CPU cores were being utilized in the training process. I would appreciate any advice on how to leverage all the CPU and GPU resources to speed up the training process. Thank you.

[glennko]

I would focus on checking on your GPU utilization more than CPUs. The CPUs aren't really used in the training process. Are you experiencing problems with the training speed?

[junyixia]

Well, I run the sample code under the Quick Start section with alpaca_data for four days and it is still not completed yet. Is it normal?

[glennko]

Did you check if your GPUs are enabled? When you import xturing, you'll get a warning if CUDA drivers are not enabled. Try torch.cuda.is_available().

[junyixia]

Yes. The GPUs are enabled with 100% usage.

[tensiondriven]

It could be that each GPU is being driven by one CPU core, hence 4 x CPU cores are running... just a thought.