Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods

By Apratim Bhattacharyya, Mario Fritz, Bernt Schiele. 2018

47 citations recorded by the source corpus.

For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence. In real-world scenarios, future states become increasingly uncertain and multi-modal, particularly on long time horizons. Dropout based Bayesian inference provides a computationally tractable, theoretically well grounded approach to learn likely hypotheses/models to deal with uncertain futures and make predictions that correspond well to observations -- are well calibrated. However, it turns out that such approaches fall short to capture complex real-world scenes, even falling behind in accuracy when compared to the plain deterministic approaches. This is because the used log-likelihood estimate discourages diversity. In this work, we propose a novel Bayesian formulation for anticipating future scene states which leverages synthetic likelihoods that encourage the learning of diverse models to accurately capture the multi-modal nature of future scene states. We show that our approach achieves accurate state-of-the-art predictions and calibrated probabilities through extensive experiments for scene anticipation on Cityscapes dataset. Moreover, we show that our approach generalizes across diverse tasks such as digit generation and precipitation forecasting.

Subfields: Artificial Intelligence, Computer Vision and Pattern Recognition, Probabilistic & Bayesian Methods, Signal Processing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Giovanni Maria Farinella
2. Antonino Furnari
3. Evangelos Kazakos
4. Alberto Garcia-Garcia
5. José A. García-Rodríguez
6. S. Orts-Escolano
7. Pablo Martinez-Gonzalez
8. John Alejandro Castro-Vargas

In Computer Vision and Pattern Recognition

1. Giovanni Maria Farinella
2. Antonino Furnari
3. Kristen Grauman
4. Dima Damen
5. Jitendra Malik
6. Luc Van Gool
7. Cordelia Schmid
8. Jonathan Munro

In Probabilistic & Bayesian Methods

1. Giovanni Maria Farinella
2. Antonino Furnari
3. Alberto Garcia-Garcia
4. José A. García-Rodríguez
5. S. Orts-Escolano
6. Pablo Martinez-Gonzalez
7. John Alejandro Castro-Vargas
8. Sergiu Oprea

In Signal Processing

1. Dima Damen
2. Giovanni Maria Farinella
3. Antonino Furnari
4. Jonathan Munro

Citations in the corpus

• Rescaling Egocentric Vision: Collection, Pipeline and Challenges for EPIC-KITCHENS-100 (2020, background): “Top-k accounts for uncertainty in future predictions, as with previous anticipation efforts [83,84,85].”
• A Review on Deep Learning Techniques for Video Prediction (2020, methodology): “In the video prediction context, VAEs are the foundation of many probabilistic models dealing with future uncertainty [9], [38], [55], [81], [85], [96], [97].”