New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cuda out of memory #37
Comments
add samples_per_gpu=1 , and issue is runtimeError: CUDA out of memory. Tried to allocate 36.00 MiB (GPU 0; 31.75 GiB total capacity; 30.02 GiB already allocated; 15.75 MiB free; 30.20 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF |
Hi, which script are you running? |
hello,mask2former_onepeace_adapter_g_896_40k_ade20k_ss.py |
When I use v100 32g to run 896896 during seg training, I will encounter obvious deficiencies. However, I still encounter this problem when I change it to 512512. Even if I change it to a smaller one, I still have it. What may be the reason for this?
The text was updated successfully, but these errors were encountered: