Skip to content

Official repository for the paper "Automating Continual Learning"

License

Notifications You must be signed in to change notification settings

IDSIA/automated-cl

Repository files navigation

Automated Continual Learning

This is the official code repository for the paper:

Automating Continual Learning

ACL overview

This codebase is originally forked from IDSIA/modern-srwm which we modified for continual learning (also including improved practical settings for self-referential weight matrices, e.g., better initialization strategy).

NB: this is research code with many sub-optimal implementations (search for NB: in main.py for various comments).

Acknowledgement

Our codebase also includes code from other public repositories, e.g.,

  • tristandeleu/pytorch-meta for standard few-shot learning data preparation/processing and data-loader implementations. (forked and slightly modified code can be found under torchmeta_local)

  • khurramjaved96/mrcl for the OML baseline (Table 3). Forked and modified code can be found under oml_baseline_local. We downloaded their out-of-the-box Omniglot model from their Google drive from the same repository.

  • GT-RIPL/Continual-Learning-Benchmark: this is not included here but we modified/used it to produce the results for the 2-task class-incremental setting (Table 3)

as well as other architectural implementations (currently not reported in the paper):

Please find LICENSE files/mentions in the corresponding directory/fileheaders.

Requirements

The basic requirements are same as the original repository IDSIA/modern-srwm/supervised_learning. We used PyTorch 1.10.2+cu102 or 1.11.0 in our experiments but newer versions should also work.

Training & Evaluation

Example training and evaluation scripts are provided under scripts. Our pre-trained model checkpoints can be downloaded from this Google drive link.

BibTex

@article{irie2023automating,
  title={Automating Continual Learning},
  author={Irie, Kazuki and Csord{\'a}s, R{\'o}bert and Schmidhuber, J{\"u}rgen},
  journal={Preprint arXiv:2312.00276},
  year={2023}
}