Taking on the curse of dimensionality in joint distributions using neural networks

By Samy Bengio, Yoshua Bengio. 2000

122 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Optimization & Training, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Yoshua Bengio
2. Mingsheng Long
3. Jianmin Wang
4. Philip S. Yu
5. Kyunghyun Cho
6. Ruslan Salakhutdinov
7. Dacheng Tao
8. Quoc V. Le

In Optimization & Training

1. Pascal Vincent
2. Yoshua Bengio
3. Hugo Larochelle
4. Iain Murray
5. Frank Hutter
6. Matthew Henderson
7. Emiel Hoogeboom
8. Auke J. Wiggers

In Signal Processing

1. Yoshua Bengio
2. Haixu Wu
3. Mingsheng Long
4. Pascal Vincent
5. Jianmin Wang
6. Iain Murray
7. Emiel Hoogeboom
8. Matthew Henderson

Citations in the corpus

• Predictive Sampling with Forecasting Autoregressive Models (2020, methodology): “An autoregressive model views x as a sequence of 1-dimensional variables ( x i ) di =1 , which suggests the following universal probability model (Bengio & Bengio, 2000; Larochelle & Murray, 2011): Universality follows from the chain rule of probability theory, and x <i denotes the values ( x j ) i…”
• Inductive biases for deep learning of higher-level cognition (2020, background): “Deep learning already benefits from a form of compositional advantage with distributed representations (Hinton, 1984; Bengio and Bengio, 2000; Bengio et al., 2001), which are at the heart of why neural networks work so well.”
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning (2021, background): “from the compositional advantage with distributed representations [75], [76].”
• A Neural Probabilistic Language Model (2003, methodology): “The idea of using neural networks to model high-dimensional discrete distributions has already been found useful to learn the joint probability of Z 1 (cid:1)(cid:1)(cid:1) Z n , a set of random variables where each is possibly of a different nature (Bengio and Bengio, 2000a,b).”
• Machine Learning for Dialog State Tracking: A Review (2015, methodology): “RNNs for tracking individual slots can be combined using a special softmax combination layer [45].”
• Quick Training of Probabilistic Neural Nets by Importance Sampling (2003, background): “These ideas have been exploited to learn the probability function of high-dimensional discrete data in (Bengio and Bengio, 2000), where comparisons have been made with polynomial learners and table-based graphical models.”
• The Neural Autoregressive Distribution Estimator (2011, background): “…a powerful framework for deriving tractable distribution estimators is the family of fully visible Bayesian networks (e.g. Frey et al., 1996; Bengio and Bengio, 2000), which decompose the observation’s probability distribution as follows: where all observation variables v i are arranged…”