An Online Spectral Learning Algorithm for Partially Observable Nonlinear Dynamical Systems

By Byron Boots, Geoffrey J. Gordon. 2011

82 citations recorded by the source corpus.

Recently, a number of researchers have proposed spectral algorithms for learning models of dynamical systems — for example, Hidden Markov Models (HMMs), Partially Observable Markov Decision Processes (POMDPs), and Transformed Predictive State Representations (TPSRs). These algorithms are attractive since they are statistically consistent and not subject to local optima. However, they are batch methods: they need to store their entire training data set in memory at once and operate on it as a large matrix, and so they cannot scale to extremely large data sets (either many examples or many features per example). In turn, this restriction limits their ability to learn accurate models of complex systems. To overcome these limitations, we propose a new online spectral algorithm, which uses tricks such as incremental Singular Value Decomposition (SVD) and random projections to scale to much larger data sets and more complex systems than previous methods. We demonstrate the new method on an inertial measurement prediction task and a high-bandwidth video mapping task and we illustrate desirable behaviors such as "closing the loop," where the latent state representation changes suddenly as the learner recognizes that it has returned to a previously known place.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

Hilbert Space Embeddings of Predictive State Representations (2013, neutral): “See, for example [4] or [20].”
Learning predictive models of a depth camera & manipulator from raw execution traces (2014, background): “Recent PSR learning algorithms rely on spectral methods [10], [27] and kernel methods [11] which are statistically consistent.”
Two Manifold Problems with Applications to Nonlinear System Identification (2012, methodology): “We propose tackling this problem by combining twomanifold methods (for task 1) with spectral learning algorithms for nonlinear dynamical systems (for task 2) (Song et al., 2010; Siddiqi et al., 2010; Boots et al., 2011; Boots & Gordon, 2010; Hsu et al., 2009; Boots et al., 2010).”
Predictive State Recurrent Neural Networks (2017, background): “By defining the belief state in terms of observables rather than latent states, PSRs can be easier to learn than other filtering methods [5, 6, 7].”
DEVELOPING A PREDICTIVE APPROACH TO KNOWLEDGE (2015, methodology): “More recently spectral learning techniques, from system identification, have been used to learn the parameters of a PSR from continuous low-level sensor data produced by a robot (Boots et al., 2010; Boots & Gordon, 2011).”