-
Notifications
You must be signed in to change notification settings - Fork 429
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to use GPU in a cloud computing environment? #1066
Comments
Hello @anderscui ! Grobid Docker image will normally work fine in a standard cloud computing environment with GPU set on the instance. Just be sure to have a proper NVIDIA GPU driver on the Linux host instance. I was able to run Grobid at scale in a few cloud environments with GPU without issue. Do you have specific examples where it does not work, apart a Lambda env.? For Amazon Lambda, I never tried it, but are you sure you can setup AWS Lambda with GPU? Maybe I misunderstand, but the idea with Lambda is that you just have a runtime environment and cannot control the infrastructure level? |
My environment is not Lambda, it's hosted in a China cloud provider. Maybe my environment is similar to this: when I deploy grobid using docker-compose, and don't add any configs specific to GPU, then grobid will only use CPU, although the host machine has an available GPU. So in this case, how can I 'tell' grobid to use the GPUs? PS: when I deploy a service (using PyTorch) in the save env, it will detect the GPUs automatically and use it. But for grobid, I think there should be some extra configs? |
I am using Docker (grobid/grobid:0.7.3). When I test on a local dedicated server, I can use
--gpus all
fordocker run
command, and it utilize the GPUs correctly.But in a cloud computing env (some service like Amazon Lambda), it seems that I cannot easily set
--gpus all
, then even the machine has GPU, the container doesn't utilize it.What can I do in this case?
What is your Java version (
java --version
)?In case of build or run errors, please submit the error while running gradlew with
--stacktrace
and--info
for better log traces (e.g../gradlew run --stacktrace --info
) or attach the log filelogs/grobid-service.log
.The text was updated successfully, but these errors were encountered: