Quan Minh Phan, Ngoc Hoang Luong
- Clone this repo:
$ git clone https://github.com/ELO-Lab/ENAS_TFI
$ cd ENAS_TFI
- Install dependencies:
$ pip install -r requirements.txt
- For single-objective NAS problems:
python main.py --n_runs 21 --warm_up 0 --nSamples_for_warm_up 0 --problem_name [problem_name] --algorithm_name GA --seed 0- For multi-objective NAS problems:
python main.py --n_runs 21 --warm_up 0 --nSamples_for_warm_up 0 --problem_name [problem_name] --algorithm_name NSGA-II --seed 0--problem_name [problem_name] receives one of following values:
| problem_name | NAS Benchmark | Type of problem | Dataset | Objecitve |
|---|---|---|---|---|
| SO-NAS101 | NAS-Bench-101 | single-objective | CIFAR-10 | validation error |
| SO-NAS201-1 | NAS-Bench-201 | single-objective | CIFAR-10 | validation error |
| SO-NAS201-2 | NAS-Bench-201 | single-objective | CIFAR-100 | validation error |
| SO-NAS201-3 | NAS-Bench-201 | single-objective | ImageNet16-120 | validation error |
| MO-NAS101 | NAS-Bench-101 | multi-objective | CIFAR-10 | #params & validation error |
| MO-NAS201-1 | NAS-Bench-201 | multi-objective | CIFAR-10 | FLOPs & validation error |
| MO-NAS201-2 | NAS-Bench-201 | multi-objective | CIFAR-100 | FLOPs & validation error |
| MO-NAS201-3 | NAS-Bench-201 | multi-objective | ImageNet16-120 | FLOPs & validation error |
To search with the Warmup method, set --warm_up 1 and set the number of samples --nSamples_for_warm_up. In our experiments, we set --nSamples_for_warm_up 500.
To experiment with the different population_size or maximum_number_of_evaluations, set another value in main.py (for population_size) and factory.py (for maximum_number_of_evaluations)
- For single-objective NAS problems:
python visualize_so.py --path_results [path_results]- For multi-objective NAS problems:
python visualize_mo.py --path_results [path_results]For example: python visualize_mo.py --path_results .\results\MO-NAS101
Note: [path_results] must only contains results of experiments are conducted on the same problem.
Our source code is inspired by:
- pymoo: Multi-objective Optimization in Python
- NSGA-Net: Neural Architecture Search using Multi-Objective Genetic Algorithm
- NAS-Bench-101: Towards Reproducible Neural Architecture Search
- NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
- How Powerful are Performance Predictors in Neural Architecture Search?