Bayesian Learning of Neural Network Architectures

By Georgi Dikov, Patrick van der Smagt, Justin Bayer. 2019

31 citations recorded by the source corpus.

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while fully stochastic networks can be more robust to parameter initialisation. The proposed method relies on standard neural variational learning and, unlike randomised architecture search, does not require a retraining of the model, thus keeping the computational overhead at minimum.

Subfields: Artificial Intelligence, Probabilistic & Bayesian Methods, Neural Architecture Search & AutoML

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Benjamin Dan Wandelt
2. Justin Alsing
3. S. Feeney
4. Wei Pan
5. Minghao Yang
6. Tom Charnock
7. Jun Wang
8. Hongpeng Zhou

In Probabilistic & Bayesian Methods

1. Benjamin Dan Wandelt
2. Justin Alsing
3. Wei Pan
4. Minghao Yang
5. S. Feeney
6. Tom Charnock
7. Jun Wang
8. Hongpeng Zhou

In Neural Architecture Search & AutoML

1. Wei Pan
2. Minghao Yang
3. Jun Wang
4. Hongpeng Zhou
5. Martin Wistuba

Citations in the corpus

• BayesNAS: A Bayesian Approach for Neural Architecture Search (2019, methodology): “Recently, Bayesian learning is applied to estimate layer size and network depth in NAS problem (Dikov et al., 2019).”
• Fast likelihood-free cosmology with neural density estimators and active learning (2019, background): “Recent developments in Bayesian inference and subsequent marginalization over network architectures may prove useful in this context (Higson et al. 2018; Dikov et al. 2019).”