-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fatal error: math.h: No such file or directory #28
Comments
Hi, Would you roll back to cuda 10.2 and try again? I have not tried cuda11, there might be unknown problems. |
I write cuda kernels because it would be a bit faster and memory efficient in some occasions. For example, when you implement label-smooth cross entropy with pytorch, you might need an one-hot tensor which requires more memory than plain label(enlarged C times, where C is number of classes). And if you would like your loss to skip ignore_labels, you would use another operator, with which you call cuda kernel again to access gpu global memory one more time, which brings some performance overheads. These problems would be partially avoided by implementing a customer cuda kernel, which in theory would save some memory and let you train your model faster. |
Hi,
I am trying to run Taylor Softmax.
(0)
I run the
python3 setup.py install
and get:I run the
python3 setup.py install
command in my dockerized research environment, which is derived from the official PyTorch GPU images:I remember when I faced similar problems in the past, I did something like this for compilation of some CUDA kernels, but then I removed these lines (it was a while ago!):
Could you maybe elaborate a bit here, since I am not very familiar with how the C++ ecosystem works.
(1)
As far as I see there is a standard autograd implementation and a custom CUDA implementation.
Since I am not very proficient with C++ and CUDA, may I ask what was the reasoning behind adding a custom CUDA kernel, was the autograd version too slow, or memory intensive?
Many thanks for you advice and code!
The text was updated successfully, but these errors were encountered: