Backprop KF: Learning Discriminative Deterministic State Estimators

By Tuomas Haarnoja, Anurag Ajay, Sergey Levine, Pieter Abbeel. 2016

228 citations recorded by the source corpus.

Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Computer Vision and Pattern Recognition

Citations in the corpus

Stochastic Prediction of Multi-Agent Interactions from Partial Observations (2019, methodology): “Our method is also related to the ”backprop Kalman ﬁltering” approach of (Haarnoja et al., 2016).”
The limits and potentials of deep learning for robotics (2018, background): “e spectrum can be achieved with differentiable versions of existing algorithms (one end of the spectrum) that are compatible solutions obtained with back-propagation (other end of the spectrum) [16], [39], [50], [108], [116], [128]. For example, Jonschkowski et al. [50] solve the aforementioned loca”
A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning (2017, background): “[11] introduces a discriminative approach to estimate the low-dimensional state of a LGSSM from input images.”
Cognitive Mapping and Planning for Visual Navigation (2017, background): “This amounts to assuming perfect visual odometry [49], which can itself be learned [25], but we defer the joint learning problem to future work.”
Global Pose Estimation with an Attention-Based Recurrent Network (2018, methodology): “Recent work includes re-formulating the localization problem as a classification task [46], a regression task [22, 44], end-to-end trainable filtering [18], and differentiable RANSAC [3].”
Disentangling Physical Dynamics From Unknown Factors for Unsupervised Video Prediction (2020, unknown): “Deep Kalman filters To handle unobserved phenomena, state space models, in particular the Kalman filter [25], have been recently integrated with deep learning, by modeling dynamics in learned latent space [28, 69, 20, 17, 3].”
Continual Learning in Deep Neural Network by Using a Kalman Optimiser (2019, methodology): “(Haarnoja et al., 2016) combine the Kalman Filter with a feed-forward Convolutional neural network.”
AI-IMU Dead-Reckoning (2019, methodology): “In this respect, our approach is related to [24] but the considered problem is more challenging: our state-space is of dimension 21 whereas [24] has a state-space of dimension only 3.”
Multiple Futures Prediction (2019, background): “The bulk of classical methods focus on using physics based dynamic or kinematic models [38, 21, 25].”
Chasing Ghosts: Instruction Following as Bayesian State Tracking (2019, background): “However, recent work has demonstrated that Bayes filter implementations – including Kalman filters [27], histogram filters [14] and particle filters [28, 29] – can be embedded into deep neural networks.”
Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation (2021, background): “Differentiable algorithms are emerging for a wide range of learning domains, including state estimation [24; 31; 34; 42], visual mapping [23; 36], planning [55; 33; 18; 49; 22; 62] and control tasks [2; 50; 17; 6].”
End-to-End Learnable Histogram Filters (2017, unknown): “And, most directly related to our work, Haarnoja et al. (2016) have applied this idea to Kalman filters, showing that measurement models based on visual input can be learned end-to-end as part of the filter.”
TRADI: Tracking deep neural network weight distributions (2019, methodology): “With the advent of DNNs, researchers have tried integrating ideas from KF in DNN training: for SLAM using RNNs [8, 16], optimization [51], DNN fusion [36].”
PVEs: Position-Velocity Encoders for Unsupervised Learning of Structured State Representations (2017, methodology): “orks [10], spatial transformer networks [5], and spatial softmax [11], which incorporate priors about visual input as architectural constraints of a neural network, as well as backprop Kalman Filters [3] and end-to-end learnable histogram ﬁlters [8], which incorporate the structure of the Bayes’ ﬁlt”
Predictive-State Decoders: Encoding the Future into Recurrent Networks (2017, background): “like a Kalman filter [22]), the equivalent objective that minimizes the negative log-likelihood is given as L = ∑ t ‖xt+1 − f(xt, ht)‖ (2).”
Predictive State Recurrent Neural Networks (2017, methodology): “[24] also recognize the complementary advantages of Bayes Filters and RNNs, and propose a new network architecture attempting to combine some of the advantages of both.”