Unsupervised learning of depth and motion

By Kishore Reddy Konda, Roland Memisevic. 2013

41 citations recorded by the source corpus.

We present a model for the joint estimation of disparity and motion. The model is based on learning about the interrelations between images from multiple cameras, multiple frames in a video, or the combination of both. We show that learning depth and motion cues, as well as their combinations, from data is possible within a single type of architecture and a single type of learning algorithm, by using biologically inspired "complex cell" like units, which encode correlations between the pixels across image pairs. Our experimental results show that the learning of depth and motion makes it possible to achieve state-of-the-art performance in 3-D activity analysis, and to outperform existing hand-engineered 3-D motion features by a very large margin.

Subfields: Biophysics, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Biophysics

In Computer Vision and Pattern Recognition

Citations in the corpus

Depth Map Prediction from a Single Image using a Multi-Scale Deep Network (2014, background): “[8], who train a factored autoencoder on image patches to predict depth from stereo sequences; however, this relies on the local displacements provided by stereo.”
FlowNet: Learning Optical Flow with Convolutional Networks (2015, methodology): “Konda and Memisevic [23] use a special autoencoder called ‘synchrony autoencoder’.”
Deep Stereo: Learning to Predict New Views from the World's Imagery (2015, methodology): “one frame). Learning has been used in several ways, including estimating the parameters of more traditional models such as MRFs [36] and learning low-level correlation ﬁlters for disparity estimation [24,17]. Unlike this prior work, we learn to synthesize new views directly using a new deep architec”
Learning to Extract Motion from Videos in Convolutional Neural Networks (2016, background): “Konda et al. [20] learned motion ﬁlters together with depth from videos.”
Visual SLAM and Structure from Motion in Dynamic Environments (2018, methodology): “The network trains the representation of motion and depth from stereo pairs by using synchrony autoencoders [77].”