Assertion failure above a certain number of models

[theorska]

Hello! I have started using taichi_three not long ago and I noticed between versions 0.0.8 and 0.0.9 the following.

I load objects to visualize different parts of a mechanical simulation. I have made these objects in Blender. Currently, I am trying to load 14+ objects with taichi_three. In 0.0.8 I was able to load 13 objects and when I tried to load in a 14th I've got the following error. I have tried the same with 0.0.9, but in that version, I am only able to load 12 objects, the 13th causes the same error.

I have tried looking at taichi documentation, but I could not find information about the taichi_max_num_snodes. Is there a way maybe to increase it? Or what does it depend on? I have also tried running it on CPU and Cuda and setting the fraction size in ti.init() but those didn't work for me.

Thank you for your answer in advance.

[Taichi] materializing...
[E 10/30/20 14:29:12.505] [struct_llvm.cpp:taichi::lang::StructCompilerLLVM::run@245] Assertion failure: (int)snodes.size() <= taichi_max_num_snodes
***********************************
* Taichi Compiler Stack Traceback *
***********************************
0x7ffa3cef2287: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3cfa3b2b: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3cee0170: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3ce222c3: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3cd935c9: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3ce8cf90: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3ce42e36: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa3cd57018: taichi::create_instance<taichi::Benchmark> in taichi_core.pyd
0x7ffa4ed796e4: PyCFunction_Call in python38.dll
0x7ffa4ed8d56d: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ecbb: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8d2e5: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed89e05: PyObject_CallFunctionObjArgs in python38.dll
0x7ffa4ed8cce0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8f0f2: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8b1bf: PyFunction_Vectorcall in python38.dll
0x7ffa4edb8cac: PyObject_FastCallDict in python38.dll
0x7ffa4edb7b09: PyObject_Call_Prepend in python38.dll
0x7ffa4edb7a79: PyCodec_EncodeText in python38.dll
0x7ffa4edcb1ff: PySequence_GetItem in python38.dll
0x7ffa4ed8fb35: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8b1bf: PyFunction_Vectorcall in python38.dll
0x7ffa4edb8c41: PyObject_FastCallDict in python38.dll
0x7ffa4edb7b09: PyObject_Call_Prepend in python38.dll
0x7ffa4edb7a79: PyCodec_EncodeText in python38.dll
0x7ffa4ed8d912: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ecbb: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4edb242b: PyEval_EvalCodeEx in python38.dll
0x7ffa4edb2389: PyEval_EvalCode in python38.dll
0x7ffa4edb2261: PyArena_New in python38.dll
0x7ffa4edb216b: PyArena_New in python38.dll
0x7ffa4ed8d0b3: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8e89f: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8d2e5: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ecbb: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8d2e5: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8f0f2: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4edb242b: PyEval_EvalCodeEx in python38.dll
0x7ffa4edb2389: PyEval_EvalCode in python38.dll
0x7ffa4edb2261: PyArena_New in python38.dll
0x7ffa4edb216b: PyArena_New in python38.dll
0x7ffa4ed8d0b3: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8e89f: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8cff0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8d2e5: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8cff0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8cff0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4ed8d2e5: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ea32: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8cff0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8e89f: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8cff0: Py_CheckFunctionResult in python38.dll
0x7ffa4ed8ecbb: PyEval_EvalFrameDefault in python38.dll
0x7ffa4ed8a258: PyEval_EvalCodeWithName in python38.dll
0x7ffa4edb242b: PyEval_EvalCodeEx in python38.dll
0x7ffa4edb2389: PyEval_EvalCode in python38.dll
0x7ffa4edb20b6: PyArena_New in python38.dll
0x7ffa4edb2045: PyArena_New in python38.dll
0x7ffa4ee64cdc: PyRun_FileExFlags in python38.dll
0x7ffa4ee64a7f: PyRun_SimpleFileExFlags in python38.dll
0x7ffa4ef22fa3: PyRun_AnyFileExFlags in python38.dll
0x7ffa4eeb6357: Py_FatalError in python38.dll
0x7ffa4ee0f29c: Py_RunMain in python38.dll
0x7ffa4ee0f125: Py_RunMain in python38.dll
0x7ffa4ee0f881: Py_Main in python38.dll
0x7ff7f32d1258: Unknown Function in python.exe
0x7ffaa8057974: BaseThreadInitThunk in KERNEL32.DLL
0x7ffaa8a8a271: RtlUserThreadStart in ntdll.dll

venv\lib\site-packages\taichi\lang\impl.py", line 228, in materialize
    taichi_lang_core.layout(layout)
RuntimeError: [struct_llvm.cpp:taichi::lang::StructCompilerLLVM::run@245] Assertion failure: (int)snodes.size() <= taichi_max_num_snodes

[archibate]

Reason:
Currently the taichi_max_num_snodes is 1024.
Each model will take up a lot of snodes for storage (each scalar field is a snode). It soon reach the limit when there're 14 objects.
The maximum number is decreasing because, there are more and more features added to taichi_three, resulting in more and more snodes-per-model.

Solution:
@yuanming-hu Maybe we should increase this limit (say, to 65536) as there are no real physics limitation there?
But even if it's possible to have more than 14 objects, the JIT compilation would be slow (isn't it?) as each model has a different render kernel to compile. I'll have a look on how to refactor taichi_three to mock this situation by spliting kernels apart. Thank for reporting!