Adding prediction pipeline

[AlexHarn]

Hi!

I am currently working on adding an inference pipeline in my own project, which is based on this template. I would be willing to see if I can add something like this to the template, especially now after we already added support for multiple pipelines. Is that something people would be interested in? @ashleve @nils-werner @charlesincharge @Steve-Tod

I know model deployment is beyond the scope of this repository, but I personally think it would still be great to have some rudimentary options to apply trained models to large data. I especially see some potential in leveraging the DDP capabilities we already have in this template.

[ashleve]

Hi @AlexHarn! I think it might be valuable to have an inference pipeline, but the idea was always to keep the template minimalistic so it's easy to digest and start a new project.

If we wanted to leverage DPP capabilities, I assume we would need to use trainer.predict() instead of model.load_from_checkpoint()? Do you think such inference pipeline would be much different than the test pipeline? It feels to me that the minimalistic example would be very similar, with the exception of using predict() instead of test().

Thanks for the idea, I'm always happy to hear propositions that make the project better.

[AlexHarn]

Yes you're right. I implemented a prediction pipeline for in my own project (kick-started with this template) and for the basic mnist example it would really just be another version of the testing pipeline except that we would call trainer.predict().

[yipliu]

Hi,

This is a fascinating question.

I also ran into problems while writing the prediction pipeline. (Data without labels)

model: LightningModule = hydra.utils.instantiate(config.model)

This line requires us to specify the model size in advance. However, I just want to load the ckpt for convenience.

[ashleve]

As far as I'm aware, the LightningModule init args get overriden by the ones stored in ckpt the moment you load it with:

model = LitModel.load_from_checkpoint(PATH)

Or with:

trainer.fit(model, datamodule, ckpt_path=...) # this reinitializes the model with correct hyperparameters

So you don't actually need to specify the model hyperparameters.

[ashleve]

You only need trainer.fit(model, datamodule, ckpt_path=...). This reinitializes model with hyperparameters stored in ckpt.

Analogically you can do trainer.predict(model, datamodule, ckpt_path=...) for inference.

[yipliu]

Hi,

Sorry for the abrupt question

I write a separate script (without the training pipeline)

If I use the training pipeline logic, I have to write the following code

model: LightningModule = hydra.utils.instantiate(config.model)
trainer.fit(model=model, datamodule=datamodule)

In this way, we have to give the concrete hparams in model.yaml.
Since I get the hparams using hyerparameter search, there are many candidates of model hparams.

So, I write the following code. The code works, but I do not know if it's right

model = LitModel.load_from_checkpoint(PATH)
trainer.predict(model, datamodule=datamodule)

In this way, I do not have to write the net concrete model hyparam in model.yaml

[ashleve]

Yes, your approach is correct.

model = MnistModel.load_from_checkpoint(PATH)
predictions = trainer.predict(model=model, datamodule=datamodule)

Btw you could also load datamodule hyperparameters from ckpt.

[yipliu]

One problem solved, a bunch of new problems popped up T-T

I post the official document here for those who need it in the future

Contents of a checkpoint

Thank you very much, very powerful template!

One more question here.

For the model, we can use load_from_checkpoint.

For datamodule hyperparameters, we just also can use load_from_checkpoint