This repository provides the implementation of the method proposed in our paper Stage-Wise Neural Architecture Search. Our method discovers competitive and high-performance architectures by exploring one order of magnitude fewer models compared to other approaches, as shown in the figure below. In addition, our method is the most resource-efficient as it designs architectures in a few hours on a single GPU.
- Scikit-learn
- Keras (Recommended version 2.1.2)
- Tensorflow (Recommended version 1.3.0 or 1.9)
- Python 3
main.py provides an example of our neural architecture search employing residual modules. We highlight that in this example, we are using only 2 epochs to discover convolutional architectures. However, in our paper, we use 200 epochs to discover the candidate architectures and each one is further trained for 100 epochs (see the experimental setup section in our paper for more details).
Our method takes two parameters:
- Number of components of Partial Least Squares (see line 313 in main.py)
The table below show the comparison between our architectures (using residual modules) with human-designed architectures. Please check our paper for more detailed results. We provide the models and weights to reproduce these results in models/CIFAR10.
| Architecture | Depth | Param. (Million) | FLOPs (Million) | Inference Time (Milliseconds) | Carbon Emission (kgCO_2eq) | Accuracy CIFAR-10 |
|---|---|---|---|---|---|---|
| ResNet56 | 56 | 0.86 | 125 | 46.86 | 1.27 | 93.03 |
| Ours (it=3) | 59 | 0.69 | 130 | 50.16 | 1.23 | 93.36 |
| ResNet110 | 110 | 1.7 | 253 | 90.33 | 2.26 | 93.57 |
| Ours (it=5) | 67 | 0.88 | 149 | 57.96 | 1.32 | 94.27 |
Please cite our paper in your publications if it helps your research.
@inproceedings{Jordao:2020,
author = {Artur Jordao,
Fernando Yamada,
Maiko Lie and
William Robson Schwartz},
title = {Stage-Wise Neural Architecture Search},
booktitle = {International Conference on Pattern Recognition (ICPR).},
}