Prune and Replace NAS
By Kevin Alexander Laube, Andreas Zell. 2019
13 citations recorded by the source corpus.
While recent Neural Architecture Search (NAS) algorithms are thousands of times faster than the pioneering works, it is often overlooked that they use fewer candidate operations, resulting in a significantly smaller search space. We present PR-DARTS, a NAS algorithm that discovers strong network configurations in a much larger search space and a single day. A small candidate operation pool is used, from which candidates are progressively pruned and replaced with better performing ones. Experiments on CIFAR-10 and CIFAR-100 achieve 2.51% and 15.53% test error, respectively, despite searching in a space where each cell has 150 times as many possible configurations than in the DARTS baseline. All of our code is available at https://github.com/cogsys-tuebingen/prdarts
Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Artificial Intelligence
In Computer Vision and Pattern Recognition
Citations in the corpus
- BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, methodology): “Some of the most popular techniques for NAS include evolutionary algorithms [47, 37], reinforcement learning [79, 42, 34, 58, 61], Bayesian optimization [21, 19, 77], gradient descent [35, 32, 29], tree search [62, 61], and neural predictors [48, 64].”
- Local Search is State of the Art for NAS Benchmarks (2020, methodology): “…techniques for NAS include Bayesian optimization (Kan-dasamy et al., 2018), reinforcement learning (Zoph and Le, 2017; Pham et al., 2018; Liu et al., 2018a; Tan and Le, 2019), gradient descent (Liu et al., 2018b; Laube and Zell, 2019), and neural predictors (White et al., 2019; Wen et al., 2019).”