Conditional Density Estimation with Bayesian Normalising Flows

By Brian L. Trippe, Richard E. Turner. 2018

104 citations recorded by the source corpus.

Modeling complex conditional distributions is critical in a variety of settings. Despite a long tradition of research into conditional density estimation, current methods employ either simple parametric forms or are difficult to learn in practice. This paper employs normalising flows as a flexible likelihood model and presents an efficient method for fitting them to complex densities. These estimators must trade-off between modeling distributional complexity, functional complexity and heteroscedasticity without overfitting. We recognize these trade-offs as modeling decisions and develop a Bayesian framework for placing priors over these conditional density estimators using variational Bayesian neural networks. We evaluate this method on several small benchmark regression datasets, on some of which it obtains state of the art performance. Finally, we apply the method to two spatial density modeling tasks with over 1 million datapoints using the New York City yellow taxi dataset and the Chicago crime dataset.

Subfields: Artificial Intelligence, Statistics and Probability

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Statistics and Probability

Citations in the corpus

Algorithmic recourse under imperfect causal knowledge: a probabilistic approach (2020, methodology): “For convenience (see §6 for details), here we opt for latent-variable implicit density models, but other conditional density estimation approaches may be also be used [e.g., 4, 6, 43].”
Analyzing Inverse Problems with Invertible Neural Networks (2018, methodology): “Besides variants of the original architectures [43, 2, 44], the most common realizations use auto-regressive flows, where the density is decomposed according to the Bayesian chain rule [23, 17, 12, 31, 29, 25, 38, 45].”
Normalizing Flows: An Introduction and Review of Current Methods (2020, background): “However, the framework was previously deﬁned in Tabak and Vanden-Eijnden [89] and Tabak and Turner [88], and explored for clustering and classiﬁcation [2], and density estimation [61], [80].”
Deep Structural Causal Models for Tractable Counterfactual Inference (2020, methodology): “These models have since been employed to capture conditional distributions [27, 29, 32, 35], and VAEs and GANs were also extended to model structured data by incorporating probabilistic graphical models [36–38].”
Low-Light Image Enhancement with Normalizing Flow (2021, methodology): “(Trippe and Turner 2018) propose to use different normalizing flows for each condition.”