Learning to Filter with Predictive State Inference Machines

By Wen Sun, Arun Venkatraman, Byron Boots, J. Bagnell. 2015

52 citations recorded by the source corpus.

Latent state space models are a fundamental and widely used tool for modeling dynamical systems. However, they are difficult to learn from data and learned models often lack performance guarantees on inference tasks such as filtering and prediction. In this work, we present the PREDICTIVE STATE INFERENCE MACHINE (PSIM), a data-driven method that considers the inference procedure on a dynamical system as a composition of predictors. The key idea is that rather than first learning a latent state space model, and then using the learned model for inference, PSIM directly learns predictors for inference in predictive state space. We provide theoretical guarantees for inference, in both realizable and agnostic settings, and showcase practical performance on a variety of simulated and real world robotics benchmarks.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

Citations in the corpus

A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning (2017, background): “Instead of basing inference on any abstract grasp of dynamics that is learned from experience, current successes are autoregressive: to imagine the tennis ball’s trajectory, one forward-generates a frame-by-frame rendering of the full sensory input [5, 7, 23, 24, 29, 30].”
Dual Policy Iteration (2018, methodology): “We believe the dependency on horizon can be brought down by leveraging system identiﬁcation methods focusing on multi-step prediction [36, 32].”
Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning (2018, methodology): “, Temporal Difference Learning (Sutton & Barto, 1998)), or V̂ e could be computed by near-optimal search algorithms via access to ground truth information (Daumé III et al., 2009; Chang et al., 2015; Sun et al., 2016) or via access to a simulator using Dynamic Programming (DP) techniques (Choudhury ”
Predictive-State Decoders: Encoding the Future into Recurrent Networks (2017, methodology): “Whereas [45] primarily studied algorithms for minimizing the predictive-state loss, we adapt it to augment general recurrent models such as LSTMs and for a wider variety of applications such as imitation and reinforcement learning.”
First-Person Activity Forecasting with Online Inverse Reinforcement Learning (2016, methodology): “More advanced methods could also be used to infer more complex transition dynamics [27, 30].”
Recurrent Environment Simulators (2017, background): “Deep neural networks have recently enabled significant advances in simulating complex environments, allowing for models that consider high-dimensional visual inputs across a wide variety of domains (Wahlström et al., 2015; Watter et al., 2015; Sun et al., 2015; Patraucean et al., 2015).”
PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations (2017, background): “For example, others train representations by using them to learn image reconstruction [21], prediction of future states [19, 21], or other auxiliary tasks [4, 12].”