-
-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
JavaScript heap out of memory during compilation of a very large model. #1071
Comments
I understand I am pushing the limits here, but at least I would like to learn why it a) ignores the |
Have you tried using CmdStan, CmdStanR, or CmdStanPy? They all use the OCaml compiler rather than cross-compiling in Javascript. It might scale a lot better. Scaling our compiler isn't something we've put much if any thought into as these large auto-generated models aren't part of our standard use case and we're stretched pretty thin on the dev side as is. Having said that, we would take contributions that scaled efficiency of our compiler, either in base OCaml or in the Javascript cross-compilation. |
Thank you very much for the answer! I tried Stan from within Python, but it failed compiling a much smaller model (it failed when the model size exceeded about 100kB). I will also try setting the |
What do you mean by "tried Stan from within Python"? We have two official R interfaces, RStan and CmdStanR, and two official Python interfaces, PyStan and CmdStanPy. CmdStanR and CmdStanPy invoke a separate process to compile a Stan program (from the makefile of CmdStan) and then a separate process to run the program, streaming results to a file. They use the direct output of compiling our OCaml transpolar. I was suggesting using one of those or using CmdStan directly to avoid Python and R altogether. RStan uses the Javascript cross-compiler of the OCaml transpiler. I don't know what PyStan does. |
Besides the RStan I used PyStan. Thank you very much again for the explanation. |
Description:
I wrote a code generator that writes a Stan code for a family of increasingly complex multinomial models.
Due to the explicit request of my employer, I am not authorized to share the actual model here. The size of the model in question is 2.3MB.
For reference, the biggest model I was able to successfully compile so far have the size of 1.6MB. It took over 8 full days to compile.
Reproducible Steps:
Current Output:
If applicable, any relevant output from RStan.
Operating System:
Ubuntu 22.04 64bit,
AMD Ryzen Threadripper 1950X 16-Core Processor
128GB of RAM
The text was updated successfully, but these errors were encountered: