Deep Neural Architecture Search with Deep Graph Bayesian Optimization

By Lizheng Ma, Jiaxu Cui, Bo Yang. 2019

57 citations recorded by the source corpus.

Image recognition aims to identify objects, places, people, or other targeted items in a given image, and has a wide range of social applications such as natural disasters recognition, plant disease detection, and traffic jam detection. Currently state-of-the-art methods of image recognition are based on deep learning and remain a common pattern in designing and using convolutional neural networks (CNNs). However, designing CNNs is extremely time intensive and requires an expert. Neural architecture search (NAS) can solve this problem by automatically identifying architectures of CNNs that are superior to hand-designed ones. Recently BO has been applied to neural architecture search and shows better performance than pure evolutionary strategies. All these methods adopt Gaussian processes (GPs) as surrogate function, with the handcraft similarity metrics as input. In this work, we propose a Bayesian graph neural network as a new surrogate, which can automatically extract features from deep neural architectures, and use such learned features to fit and characterize black-box objectives and their uncertainty. Based on the new surrogate, we then develop a graph Bayesian optimization framework to address the challenging task of deep neural architecture search. Experiment results show our method significantly outperforms the comparative methods on benchmark tasks. CCS CONCEPTS • Networks → Network performance evaluation; • Computing methodologies → Artificial intelligence.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search (2019, methodology): “The two best-performing neural predictors are an ensemble of GCNs, and an ensemble of feedforward neural networks with the path encoding, with the latter having a slight edge.”
Neural Architecture Search with Reinforce and Masked Attention Autoregressive Density Estimators (2020, background): “, 2018), Bayesian optimization (Shahriari et al., 2015; Kandasamy et al., 2018; Ma et al., 2019; White et al., 2019), evolutionary learning (Real et al.”