The informed sampler: A discriminative approach to Bayesian inference in generative computer vision models

By Varun Jampani, Sebastian Nowozin, Matthew Maverick Loper, Peter Gehler. 2014

46 citations recorded by the source corpus.

The informed sampler - a general inference technique for Bayesian posterior inference in generative models.This method leverages discriminative computer vision models for faster probabilistic inference in generative models.Three different applications that highlight common challenges of posterior inference.Detailed comparisons and analysis with respect to different baseline sampling based methods.Informed sampling is found to converge faster than all baseline samplers across diverse problems. Computer vision is hard because of a large variability in lighting, shape, and texture; in addition the image signal is non-additive due to occlusion. Generative models promised to account for this variability by accurately modelling the image formation process as a function of latent variables with prior beliefs. Bayesian posterior inference could then, in principle, explain the observation. While intuitively appealing, generative models for computer vision have largely failed to deliver on that promise due to the difficulty of posterior inference. As a result the community has favoured efficient discriminative approaches. We still believe in the usefulness of generative models in computer vision, but argue that we need to leverage existing discriminative or even heuristic computer vision methods. We implement this idea in a principled way with an informed sampler and in careful experiments demonstrate it on challenging generative models which contain renderer programs as their components. We concentrate on the problem of inverting an existing graphics rendering engine, an approach that can be understood as "Inverse Graphics". The informed sampler, using simple discriminative proposals based on existing computer vision technology, achieves significant improvements of inference.

Subfields: Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

Citations in the corpus

Towards causal generative scene models via competition of experts (2020, methodology): “Furthermore, gains achieved by a more accurate scene generation process are generally paid for by complicated inference, and most methods thus rely on variations of MCMC sampling schemes (Jampani et al., 2015; Wu et al., 2017).”
Attend, Infer, Repeat: Fast Scene Understanding with Generative Models (2016, methodology): “Hinton et al. (1995); Tu & Zhu (2002); Kulkarni et al. (2015a); Jampani et al. (2015); Wu et al. (2015) exploit data-driven predictions to empower the ‘vision as inverse graphics’ paradigm.”
Neural Scene De-rendering (2017, methodology): “Our framework also relates to the field of “vision as inverse graphics”, analysis-by-synthesis, or generative models with data-driven proposals [36, 37, 30, 20, 34, 16], as we are incorporating a graphics engine as a black-box synthesizer.”
Synthesizing Programs for Images using Reinforced Adversarial Learning (2018, background): “The idea of inverting simulators to interpret images has been explored extensively in recent years (Nair et al., 2008; Paysan et al., 2009; Mansinghka et al., 2013; Loper & Black, 2014; Kulkarni et al., 2015a; Jampani et al., 2015).”
Applications of Probabilistic Programming (Master's thesis, 2015) (2016, background): “In addition, to improve general-purpose message passing in Infer.NET for graphical models with many layers, which are often used in computer vision, a consensus message passing method has been proposed in (Jampani et al., 2015a).”
DeepCoder: Learning to Write Programs (2016, unknown): “Examples include Dayan et al. (1995); Kingma & Welling (2014); Shotton et al. (2013); Stuhlmüller et al. (2013); Heess et al. (2013); Jampani et al. (2015).”
OpenDR: An Approximate Differentiable Renderer (2014, background): “Consequently, differentiable rendering is only one piece of the puzzle: we believe that informed sampling [15] and probabilistic graphics programming [22] are also essential to a serious application of inverse rendering.”
Inference Networks for Sequential Monte Carlo in Graphical Models (2016, background): “…models in which closed-form full conditionals are not available (Stuhlmüller et al., 2013), using pre-trained neural networks to inform local MCMC proposal kernels (Jampani et al., 2015; Kulkarni et al., 2015), and learning messages for new factors for expectation-propagation (Heess et al., 2013).”
Gen: a general-purpose probabilistic programming system with programmable inference (2019, methodology): “An emerging approach to inference in these models uses ‘data-driven’ custom proposal distributions trained using supervised learning on joint samples from the generative model to accelerate Monte Carlo algorithms [30, 35].”
Data-driven Sequential Monte Carlo in Probabilistic Programming (2015, methodology): “The work on using discriminative proposals for Markov Chain Monte Carlo in parametric generative models include [8] and [4], with applications in computer vision.”
Deep Generative Vision as Approximate Bayesian Computation (2014, background): “Probabilistic formulations of inverse graphics have recently been proposed for a variety of 2D and 3D vision problems [15, 12, 14, 9].”
Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods (2019, background): “We also note that a number of inverse graphics models [85], [86], [87] have been proposed to train disentangled representations.”