Adding sequential inference

[ancestor-mithril]

Changing the implementation of predict_single_npy_array from parallel to sequential.

By default, using predict_single_npy_array uses PreprocessAdapterFromNpy which creates an additional process for loading the data. This process is not needed because the data was already read into a npy array and is passed to nnUNet. This is also faster because Tensors do not need to be transferred between processes.
predict_single_npy_array does not benefit from parallelism in any situation and the users would benefit more for being able to parallelize themselves predictor.predict_single_npy_array.

Adding support for sequential prediction from files

This allows running nnUNetv2_predict with a single process. This allows using less RAM usage per nnUNetv2_predict and better parallelization across GPUs when running multiple nnUNetv2_predict.

Previous behavior

To reduce RAM usage when doing prediction you could use nnUNetv2_predict ... -npp 1 -nps 1.

Current behavior

Now you can use nnUNetv2_predict ... -npp 0 -nps 0. It uses less RAM. It is faster when used only for 1 file or parallelized across many GPUs with -num_parts X.

[FabianIsensee]

I am confused. predictor.predict_single_npy_array does not spawn any processes. It all runs in the main process. Can you point me to the place where multiprocessing is supposedly used?

[ancestor-mithril]

Hi, sorry for the confusion, I may be wrong about it.
I know that predict_from_files definitely uses multiple processes for loading the data and exporting the segmentation.
I think the reason of my confusion was that predictor.predict_single_npy_array uses PreprocessAdapterFromNpy, inheriting from the batchgenerators.dataloading.data_loader.DataLoader.
The PyTorch Dataloader uses 1 additional process when num_workers=1 and runs in the main process when num_workers=0, while the batchgenerators DataLoader does not have multiprocessing built within itself.

[FabianIsensee]

Exactly, so the current code is already sequential

[ancestor-mithril]

Yes, you are right. Only the part about predict_from_files is relevant. If you are interested in a sequential implementation for predict_from_files (nnUNetv2_predict), I can draft a new PR with only this part.