An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models

By Harri Valpola, Juha Karhunen. 2002

180 citations recorded by the source corpus.

Subfields: Artificial Intelligence, Control and Systems Engineering, Probabilistic & Bayesian Methods, Signal Processing, Time-Series & Forecasting

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Control and Systems Engineering

In Probabilistic & Bayesian Methods

In Signal Processing

In Time-Series & Forecasting

Citations in the corpus

Density estimation using Real NVP (2016, neutral): “However, as a naive application of this formula is in general impractical for highdimensional data, ICA practitioners preferred to use more tractable principle like ensemble learning [58].”
How to Construct Deep Recurrent Neural Networks (2013, methodology): “Earlier, Valpola and Karhunen (2002) used a deep neural network to model the state transition in a nonlinear, dynamical state-space model.”
On the challenges of learning with inference networks on sparse, high-dimensional data (2017, background): “The assumption of linearity has been relaxed in nonlinear factor analysis (NFA) (Gibson, 1960) and extended across a variety of domains such as economics (Jones, 2006), signal processing (Jutten & Karhunen, 2003), and machine learning (Valpola & Karhunen, 2002; Lawrence, 2003).”
Deep Kalman Filters (2015, methodology): “They approximate the posterior using non-linear dynamic factor analysis [18], which scales quadratically with the latent dimension.”
Structured Inference Networks for Nonlinear State Space Models (2016, background): “They approximate the posterior with non-linear dynamic factor analysis (Valpola and Karhunen 2002), which scales quadratically with the observed dimension and is impractical for large-scale learning.”
DEM: A variational treatment of dynamic systems (2008, methodology): “These schemes solve a dual-estimation problem; for example, Valpola and Karhunen (2002) describe a detailed and general approach to ensemble learning for nonlinear state-space models.”
Natural Conjugate Gradient in Variational Inference (2007, methodology): “To study the performance differences between the natural conjugate gradient (NCG) method, the conjugate gradient (CG) method and the heuristic algorithm from (Valpola and Karhunen, 2002), the algorithms were applied to different sized parts of the speech data set.”
Advances in Nonlinear Blind Source Separation (2003, methodology): “The model (34)– (35) is discussed in detail in [61].”
Nonlinear ICA of Temporally Dependent Stationary Sources (2017, background): “Recent deep learning research (Mobahi et al., 2009; Springenberg and Riedmiller, 2012; Goroshin et al., 2015) as well as blind source separation research (Valpola and Karhunen, 2002; Hosseini and Deville, 2014) use similar ideas.”
Adaptive BCI based on variational Bayesian Kalman filtering: an empirical evaluation (2004, methodology): “Contrary to previous use of variational methods for general state space models [34], [35], our approach tracks the parameters of a classifier and variational learning infers the model parameters and the precision of the process noise .”
Variational Bayesian learning of nonlinear hidden state-space models for model predictive control (2009, methodology): “For OIC, the same reference signal was used as in NMPC. Means of the states were approximated with an extended Kalman smoother and these were then used to initialise the NDFA alongside with variances from the previous iteration.”
Unified Inference for Variational Bayesian Linear Gaussian State-Space Models (2006, methodology): “Here we take instead the ML-II (‘evidence’) framework, in w hich the optimal set of hyperparameters is found by maximizing p(v1:T |Θ̂) with respect tôΘ [6, 7, 9].”
On-line Variational Bayesian Learning (2003, methodology): “Additionally the method could be applied to other previous models such as [2, 10, 3, 14, 17, 11, 20].”
Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes (2010, methodology): “…differences between the Riemannian conjugate gradient (RCG) method, standard Riemannian gradient (RG) method, standard conjugate gradient (CG) method and the heuristic algorithm from Valpola and Karhunen (2002), the algorithms were applied to different sized parts of the speech data set.”