Variational Inference for Large-Scale Models of Discrete Choice

By Michael Braun, Jon D. McAuliffe. 2007

193 citations recorded by the source corpus.

Discrete choice models are commonly used by applied statisticians in numerous fields, such as marketing, economics, finance, and operations research. When agents in discrete choice models are assumed to have differing preferences, exact inference is often intractable. Markov chain Monte Carlo techniques make approximate inference possible, but the computational cost is prohibitive on the large datasets now becoming routinely available. Variational methods provide a deterministic alternative for approximation of the posterior distribution. We derive variational procedures for empirical Bayes and fully Bayesian inference in the mixed multinomial logit model of discrete choice. The algorithms require only that we solve a sequence of unconstrained optimization problems, which are shown to be convex. One version of the procedures relies on a new approximation to the variational objective function, based on the multivariate delta method. Extensive simulations, along with an analysis of real-world data, demonstrate that variational methods achieve accuracy competitive with Markov chain Monte Carlo at a small fraction of the computational cost. Thus, variational methods permit inference on datasets that otherwise cannot be analyzed without possibly adverse simplifications of the underlying discrete choice model. Appendices C through F are available as online supplemental materials.

Subfields: Artificial Intelligence, Economics and Econometrics, Marketing

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

In Economics and Econometrics

In Marketing

Citations in the corpus

Variational Inference: A Review for Statisticians (2016, methodology): “In other work, Braun and McAuliffe (2010) derived a variational inference algorithm for the discrete choice model, which also lies outside of the class of conditionally conjugate models.”
Relational Topic Models for Document Networks (2009, unknown): “We use a first-order approximation (Braun and McAuliffe 2007), L d1,d2 ≡ E q [log p(y d1,d2 = 1|z d1 , z d2 , η, ν)] ≈ η T π d1,d2 + ν + log σ −η T π d1,d2 − ν , (5) where π d1,d2 = φ d1 • φ d2 and φ d = E q [z d ] =”
Celeste: Variational inference for a generative model of astronomical images (2015, methodology): “This technique is known as delta-method variational inference (Braun & McAuliffe, 2010; Wang & Blei, 2013).”
Online Learning for Latent Dirichlet Allocation (2010, methodology): “Although the choice of approximate posterior introduces bias, VB is empirically shown to be faster than and as accurate as MCMC, which makes it an attractive option when applying Bayesian models to large datasets [1, 2, 3].”
Bayesian forecasting in economics and finance: A modern review (2022, background): “Most notably, for problems in which the dimension of the unknowns, and possibly that of y1:T also, is large, the production of q(θ) is much faster (often orders of magnitude so) than producing an estimate of p(θ|y1:T ) (and any associated quantities, including predictives) via simulation (Braun and ”
Decomposing Spatiotemporal Brain Patterns into Topographic Latent Sources (2014, unknown): “However, linearization often works quite well in practice (Braun and McAuliffe, 2010; Chappell et al., 2009; Friston et al., 2007; Gershman et al., 2012), and we confirm this empirically below.”
Mixed Membership Stochastic Blockmodels (2007, background): “and Braun and McAuliffe (2007). We refer readers interested in the comparison between variational vs.”
Nonparametric variational inference (2012, methodology): “The approximate expectation is This approximation is known as the multivariate delta method for moments (Bickel & Doksum, 2007), and is often used within variational inference schemes for models that cannot exploit conjugacy (e.g., Braun & McAuliﬀe, 2010).”
Fully simplified multivariate normal updates in non-conjugate variational message passing (2014, unknown): “Others include Braun and McAuliffe (2010), Wand et al. (2011) and Wang and Blei (2013). Section 2 lays out notation needed for the main theorems, which are presented in Section 4.”
Marketing Analytics for Data-Rich Environments (2016, unknown): “A related technique that uses tractable deterministic approximations to the posterior distribution is variational inference (Braun and McAuliffe 2010).”
Variational learning for latent Gaussian model of discrete data (2012, methodology): “This can also be interpreted as the choices made by an “average agent” [Braun and McAuliffe, 2010].”
Mean field variational bayes for elaborate distributions (2011, methodology): “Recently, Braun and McAuliﬀe (2010) used yet another approach, a version of the multivariate delta method, to handle elaborate distributions arising in discrete choice models.”
Black Box Variational Inference (2013, methodology): “Deriving these algorithms on a model-by-model basis is tedious work.”
Understanding Large-Scale Dynamic Purchase Behavior (2020, unknown): “To date, VI has been adopted in few marketing papers (Braun and McAuliffe 2010, Dzyabura and Hauser 2011, Ansari et al. 2018, Xia et al. 2019).”