Cubic-Spline Flows

By Conor Durkan, Artur Bekasov, Iain Murray, George Papamakarios. 2019

63 citations recorded by the source corpus.

A normalizing flow models a complex probability density as an invertible transformation of a simple density. The invertibility means that we can evaluate densities and generate samples from a flow. In practice, autoregressive flow-based models are slow to invert, making either density estimation or sample generation slow. Flows based on coupling transforms are fast for both tasks, but have previously performed less well at density estimation than autoregressive flows. We stack a new coupling transform, based on monotonic cubic splines, with LU-decomposed linear layers. The resulting cubic-spline flow retains an exact one-pass inverse, can be used to generate high-quality images, and closes the gap with autoregressive flows on a suite of density-estimation tasks.

Subfields: Computational Mechanics, Generative Models, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

1. George Papamakarios
2. Iain Murray
3. Shakir Mohamed
4. Conor Durkan
5. Artur Bekasov
6. Josh Tenenbaum
7. Jiajun Wu

In Generative Models

1. Balaji Lakshminarayanan
2. George Papamakarios
3. Shakir Mohamed
4. Danilo Jimenez Rezende
5. Conor Durkan
6. Stefano Ermon
7. Eric T. Nalisnick
8. Iain Murray

In Probabilistic & Bayesian Methods

1. Danilo Jimenez Rezende
2. George Papamakarios
3. Conor Durkan
4. Balaji Lakshminarayanan
5. Shakir Mohamed
6. Iain Murray
7. Eric T. Nalisnick
8. Max Welling

Citations in the corpus

• Neural Spline Flows (2019, methodology): “Cubic splines In a previous iteration of this work [10], we explored the cubic-spline flow, a natural extension to the framework of Müller et al.”
• Normalizing Flows for Probabilistic Modeling and Inference (2019, methodology): “Whitening transformations (Johnson, 1966; Friedman, 1987)—so-called as they transform data into white noise—are the clearest intellectual predecessor to the use of normalizing flows within machine learning. Chen and Gopinath (2000) were perhaps the first to use whitening as a density estimation tech”
• Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models (2021, unknown): “Linear and quadratic splines [137], cubic splines [45], and rational-quadratic splines [46] have been applied so far.”