-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
DL2 OOM issues #70
Comments
I had success with 70G for the Crab models and already a commit to fix it in #67. Need to check though whether the Crab models are the largest and change accordingly. Also might calculate it on the fly, should be possible I think. Then even smaller models get scheduled quicker. |
For now Fixed via #72, right? |
Yeah. I left it open as thats not a nice or long-term solution, but if you want to we can close the issue for now |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The lstchain dl2 script uses enormous amounts of memory due to loading all models and data into ram at once. Nothing we can really do about that here. We had oom issues even with 64G due to the larger models on the crab declination line, so we probably need to increase even further.
Might slow things down on the slurm side, but this is really a (known) lstchain problem.
The text was updated successfully, but these errors were encountered: