PreNAS is a novel learning paradigm that integrates one-shot and zero-shot NAS techniques to enhance search efficiency and training effectiveness. This search-free approach outperforms current state-of-the-art one-shot NAS methods for both Vision Transformer and convolutional architectures, as confirmed by its superior performance when the code is released.

Wang H, Ge C, Chen H and Sun X. PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search. ICML 2023.

Paper link: arXiv

To set up the environment you can easily run the following command:

conda create -n PreNAS python=3.7
conda activate PreNAS
pip install -r requirements.txt

You need to download the ImageNet-2012 to the folder ../data/imagenet.

The code was run on 8 x 80G A100.

• Zero-Shot Search

  bash 01_zero_shot_search.sh

• One-Shot Training

  bash 02_one_shot_training.sh

• Evaluation

  bash 03_evaluation.sh

If PreNAS is useful for you, please consider to cite it. Thank you! :)

@InProceedings{PreNAS,
    title     = {PreNAS: Preferred One-Shot Learning Towards Efficient Neural Architecture Search},
    author    = {Wang, Haibin and Ge, Ce and Chen, Hesen and Sun, Xiuyu},
    booktitle = {International Conference on Machine Learning (ICML)},
    month     = {July},
    year      = {2023}
}

The codes are inspired by AutoFormer.