Towards Bayesian Deep Learning: A Survey

By Hao Wang, Dit-Yan Yeung. 2016

49 citations recorded by the source corpus.

While perception tasks such as visual object recognition and text understanding play an important role in human intelligence, the subsequent tasks that involve inference, reasoning and planning require an even higher level of intelligence. The past few years have seen major advances in many perception tasks using deep learning models. For higher-level inference, however, probabilistic graphical models with their Bayesian nature are still more powerful and flexible. To achieve integrated intelligence that involves both perception and inference, it is naturally desirable to tightly integrate deep learning and Bayesian models within a principled probabilistic framework, which we call Bayesian deep learning. In this unified framework, the perception of text or images using deep learning can boost the performance of higher-level inference and in return, the feedback from the inference process is able to enhance the perception of text or images. This survey provides a general introduction to Bayesian deep learning and reviews its recent applications on recommender systems, topic models, and control. In this survey, we also discuss the relationship and differences between Bayesian deep learning and other related topics like Bayesian treatment of neural networks.

Subfields: Artificial Intelligence

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. P. Heng
2. Alun David Preece
3. Qi-Yan Dou
4. Hao Chen
5. Jing Qin
6. Lequan Yu
7. Federico Cerutti
8. Dave Braines

Citations in the corpus

• Deep Learning for Health Informatics (2017, background): “Bayesian networks [38], [39] are a particular case of network”
• Asking 'Why' in AI: Explainability of intelligent systems - perspectives and challenges (2018, background): “Recent work aiming to integrate deep learning and Bayesian models within a uniform probabilistic framework appears promising in this context (Wang and Yeung, 2016).”
• Survey on Deep Neural Networks in Speech and Vision Systems (2019, background): “Some of the solutions such as data augmentation [148, 149], transfer learning [150], and introduction of Bayesian concepts [151, 152] have laid the groundwork for using small data, which we expect to progress over time.”
• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges (2020, methodology): “this issue, BNNs/BDL [24], [25], [26] can be used to interpret the model parameters.”
• Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications (2018, background): “, GAN [13] and deep Bayesian network [4, 35].”
• Automated Melanoma Recognition in Dermoscopy Images via Very Deep Residual Networks (2017, methodology): “In the future, we shall investigate to integrate Bayesian learning, especially probabilistic graphical models [48], [49]”
• Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges (2017, methodology): “with PGM and latent variable models forms a probabilistic frameworkwhere deep learning is used as a substrate formaking improved learning architecture for recommender systems, topic modeling, and control systems [105].”
• Interpretability of deep learning models: A survey of results (2017, background): “exploited in recent work on Bayesian Deep Learning (BDL) in particular [49] aiming to integrate deep learning and Bayesian”