Long-Term Image Boundary Extrapolation

By Apratim Bhattacharyya, Mateusz Malinowski, Bernt Schiele, Mario Fritz. 2016

12 citations recorded by the source corpus.

Boundary prediction in images and videos has been a very active topic of research and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception. While prior work has focused on predicting boundaries for observed frames, our work aims at predicting boundaries of future unobserved frames. This requires our model to learn about the fate of boundaries and extrapolate motion patterns. We experiment on established real-world video segmentation dataset, which provides a testbed for this new task. We show for the first time spatio-temporal boundary extrapolation, that in contrast to prior work on RGB extrapolation maintains a crisp result. Furthermore, we show long-term prediction of boundaries in situations where the motion is governed by the laws of physics. We argue that our model has with minimalistic model assumptions derived a notion of"intuitive physics".

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Visual Interaction Networks: Learning a Physics Simulator from Video (2017, methodology): “A third class of methods [5, 8, 9, 23], like our Visual Interaction Network, have been used to predict future state descriptions from pixels.”
Relational inductive bias for physical construction in humans and machines (2018, background): “…while much of the work on physical reasoning has focused on the task of prediction (e.g. Fragki-adaki et al., 2016; Mottaghi, Bagherinezhad, et al., 2016; Mottaghi, Rastegari, et al., 2016; Stewart & Ermon, 2017; Bhattacharyya et al., 2018) or inference (e.g. Wu et al., 2016; Denil et al., 2017).”