Deep Active Learning with a Neural Architecture Search

By Yonatan Geifman, Ran El-Yaniv. 2018

50 citations recorded by the source corpus.

We consider active learning of deep neural networks. Most active learning works in this context have focused on studying effective querying mechanisms and assumed that an appropriate network architecture is a priori known for the problem at hand. We challenge this assumption and propose a novel active strategy whereby the learning algorithm searches for effective architectures on the fly, while actively learning. We apply our strategy using three known querying techniques (softmax response, MC-dropout, and coresets) and show that the proposed approach overwhelmingly outperforms active learning using fixed architectures.

Subfields: Artificial Intelligence, Computer Networks and Communications, Neural Architecture Search & AutoML, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Martin Mundt
2. Izhak Golan
3. Yongjun Hong
4. Visvanathan Ramesh
5. Iuliia Pliushch
6. Po-Yao (Bernie) Huang
7. Xiaojun Chang
8. Zhihui Li

In Computer Networks and Communications

1. Iuliia Pliushch
2. Visvanathan Ramesh
3. Yongjun Hong
4. Martin Mundt
5. Ran El-Yaniv
6. Xiaojiang Chen
7. Izhak Golan
8. Yedid Hoshen

In Neural Architecture Search & AutoML

1. Iuliia Pliushch
2. Visvanathan Ramesh
3. Yongjun Hong
4. Martin Mundt
5. Ran El-Yaniv
6. Xiaojun Chang
7. Xin Wang
8. Po-Yao (Bernie) Huang

In Statistical Learning Theory

1. Iuliia Pliushch
2. Visvanathan Ramesh
3. Yongjun Hong
4. Martin Mundt
5. Ran El-Yaniv
6. Xiaojun Chang
7. Xin Wang
8. Po-Yao (Bernie) Huang

Citations in the corpus

• A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning (2020, methodology): “Here, simply using neural network prediction confidence, predictive entropy or other derived heuristics (Lewis and Gale, 1994) are still practically employed in comparisons today (Geifman and El-Yaniv, 2019).”
• Deep Anomaly Detection Using Geometric Transformations (2018, background): “Some examples are open-world classification, selective classification and regression [36, 35, 13], uncertainty estimation [15], and deep active learning [12, 14].”
• A Survey of Deep Active Learning (2020, methodology): “g to dynamically adjust the acquisition function so as to obtain high-quality query samples. Fig.3 presents a comparison between traditional AL, RAL and DRAL pipelines. On the other hand, Active-iNAS [50] notices that most previous DAL methods [3, 5, 89] assume that a suitable DL model has been desi”