Mixture density networks

By Charles M. Bishop. 1994

Subfields: Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Probabilistic & Bayesian Methods

Citations in the corpus

Neural Autoregressive Distribution Estimation (2016, methodology): “The resulting model can be interpreted as a sequence of mixture density networks (Bishop, 1994) with shared parameters.”
HyperNetworks (2016, background): “…already does a formidable job of generating time-varying parameters of a Mixture Gaussian distribution used to generate realistic handwriting samples, the ability to go one level deeper, and to dynamically generate the generative model is one of the key advantages of HyperRNN over a normal RNN.”
Hierarchical Multiscale Recurrent Neural Networks (2016, methodology): “is experiment, we use the same model as the one used in the character-level language model, except that the output layer is modiﬁed to predict real-valued outputs. We use the mixture density network (Bishop, 1994) as the output layer following Graves (2013), and use 400 units for each HM-LSTM layer ”
A Deep and Tractable Density Estimator (2013, methodology): “NADE has recently been extended to allow density estimation of real-valued vectors (Uria et al., 2013) by using mixture density networks or MDNs (Bishop, 1994) for each of the conditionals in Equation (1).”
A Meta-Transfer Objective for Learning to Disentangle Causal Mechanisms (2019, methodology): “We model the conditionals with mixture density networks (Bishop, 1994) and the marginals by a Gaussian mixture.”
Word Representations via Gaussian Embedding (2014, methodology): “This allows us to go beyond the Bayesian approach and use arbitrary (and even asymmetric) training criteria, and is more similar to methods that learn kernels (Lanckriet et al., 2004) or function-valued neural networks such as mixture density networks (Bishop, 1994).”
A Neural Representation of Sketch Drawings (2017, methodology): “In Equation 4, we model (∆x,∆y) as a Gaussian Mixture Model (GMM) with M normal distributions as in [2, 8], and (q1, q2, q3) as a categorical distribution to model the ground truth data (p1, p2, p3), where (q1 + q2 + q3 = 1) as done in [9] and [31].”
MelNet: A Generative Model for Audio in the Frequency Domain (2019, methodology): “Following the work on Mixture Density Networks [4] and their application to autoregressive models [15], θij is”
Prosody Learning Mechanism for Speech Synthesis System Without Text Length Limit (2020, methodology): “Considering the variability of prosody, the mix density network [32] is used to model the distribution of the prosody representation.”
Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows (2018, methodology): “In all experiments, SNPE-A uses a Mixture Density Network [7] with two hidden layers of 50 units each, 8 mixture components, and tanh nonlinearities.”
Regularized Hierarchical Policies for Compositional Transfer in Robotics (2019, unknown): “rference of opposing modes of behaviour. With this work we continue to explore the beneﬁts of hierarchy for multitask learning and the modelling of multi-modal distributions [Rosenstein et al., 2005, Bishop, 1994]. We focus on developing an actor-critic RL approach for training hierarchical, modular”
Likelihood-free MCMC with Amortized Approximate Ratio Estimators (2019, methodology): “es. Similarly, approaches like Sequential Neural Posterior Estimation (SNPE) [10] borrow ideas from VI literature and iteratively adjusts a parameterized posterior deﬁned as a Mixture Density Network [48]. Instead of learning the posterior directly, Sequential Neural Likelihood (SNL) [8] uses modern”
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale (2019, methodology): “mbeddings of size 4 given by single-layer NNs; and address embeddings of size 64. The proposal layers are two-layer NNs, the output of which are either a mixture of ten truncated normal distributions [12] (for uniform continuous priors) or a categorical distribution (for categorical priors). We use ”
Causal feature learning: an overview (2017, unknown): “In very high-dimensional problems, an MDN with several mixture components might be unable to learn the true density accurately.”
Deep Learning for Classical Japanese Literature (2018, methodology): “anjiVG (Modern Kanji), and a Sketch-RNN model to generate stroke versions of KanjiVG conditioned on the latent space, z new. (2) We train a VAE on Kuzushiji-Kanji, and train a Mixture Density Network [2] to predict P(z newjz old). We generate stroke versions of Modern Kanji based on the predicted ze”
State Space LSTM Models with Particle MCMC Inference (2017, methodology): “ma & Welling, 2014; Rezende et al., 2014). Enriching the output distribution of RNN has also regain popularity recently. Unlike conventionally used multinomial output or mixture density networks (Bishop, 1994), recent approaches seek for more ﬂexible family of distributions such as restricted Bo”
Exploring Interpretable LSTM Neural Networks over Multi-Variable Data (2019, background): “enriched hidden state of each variable, variable attention is derived to merge variable-wise states. These two steps are assembled into a probabilistic mixture model (Zong et al., 2018; Graves, 2013; Bishop, 1994), which facilitates the subsequent learning, predicting, and interpreting. In particula”
Siri On-Device Deep Learning-Guided Unit Selection Text-to-Speech System (2017, methodology): “As a combination of a conventional neural network with a probability distribution, MDN provides a framework to model conditional density functions [10, 11].”
Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model (2018, background): “[21] C.”