-
Notifications
You must be signed in to change notification settings - Fork 91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
an illegal memory access was encountered #39
Comments
@FishBigOcean Were you able to solve this? |
hello!When i use multi gpu, I also have the same problem,were you able to solve it? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
out of memory
invalid argument
an illegal memory access was encountered
an illegal memory access was encountered
2019-08-12 01:46:36.265776: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:639] failed to record completion event; therefore, failed to create inter-stream dependency
2019-08-12 01:46:36.265776: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:639] failed to record completion event; therefore, failed to create inter-stream dependency
2019-08-12 01:46:36.265818: I tensorflow/stream_executor/stream.cc:4138] stream 0x5580c77c5e60 did not memcpy host-to-device; source: 0x7fbeda64a010
2019-08-12 01:46:36.265800: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:639] failed to record completion event; therefore, failed to create inter-stream dependency
2019-08-12 01:46:36.265814: I tensorflow/stream_executor/stream.cc:4138] stream 0x5580c77c5e60 did not memcpy host-to-device; source: 0x1020dc1a600
2019-08-12 01:46:36.265851: E tensorflow/stream_executor/stream.cc:289] Error recording event in stream: error recording CUDA event on stream 0x5580c77c5f30: CUDA_ERROR_ILLEGAL_ADDRESS; not marking stream as bad, as the Event object may be at fault. Monitor for further errors.
2019-08-12 01:46:36.265855: I tensorflow/stream_executor/stream.cc:4138] stream 0x5580c77c5e60 did not memcpy host-to-device; source: 0x1020dc26200
2019-08-12 01:46:36.265872: E tensorflow/stream_executor/cuda/cuda_event.cc:49] Error polling for event status: failed to query event: CUDA_ERROR_ILLEGAL_ADDRESS
2019-08-12 01:46:36.265886: F tensorflow/core/common_runtime/gpu/gpu_event_mgr.cc:203] Unexpected Event status: 1
已放弃 (核心已转储)
I have encountered this problem, my environment is installed according to the readme, is there anyone who has a solution?
The text was updated successfully, but these errors were encountered: