Statistical Model Criticism using Kernel Two Sample Tests

By James Robert Lloyd, Zoubin Ghahramani. 2015

93 citations recorded by the source corpus.

Subfields: Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “criticism (Lloyd and Ghahramani 2015, Kim et al. 2016), approximate Bayesian computation (Park et al. 2016), and model selection in generative models (Bounliphone et al. 2016), for example.”
Interpretable Distribution Features with Maximum Testing Power (2016, methodology): “Lloyd and Ghahramani (2015) used this witness function to compare the model output of the Automated Statistician (Lloyd et al., 2014) with a reference sample, yielding a visual indication of where the model fails.”
A Test of Relative Similarity For Model Selection in Generative Models (2015, methodology): “…Dziugaite et al., 2015), where the MMD measures the distance of the generated samples to some reference target set; it has been used for statistical model criticism (Lloyd and Ghahramani, 2015); and to minimize the effect of nuisance variables on learned representations (Louizos et al., 2016).”
Fast Two-Sample Testing with Analytic Representations of Probability Measures (2015, methodology): “These include data integration in bioinformatics [6], benchmarking for steganography [20] and automated model checking [19].”
Revisiting Classifier Two-Sample Tests (2016, background): “Over the last century, statisticians have nurtured a wide variety of two-sample tests.”
Making Tree Ensembles Interpretable: A Bayesian Model Selection Approach (2016, methodology): “For instance, we may be able to design a better prediction model by fixing the bug [14] in the model [15], or we can make a better decision based on the insights on the model [16, 13].”
Near-linear time approximation algorithms for optimal transport via Sinkhorn iteration (2017, background): “Computing distances between probability measures on metric spaces, or more generally between point clouds, plays an increasingly preponderant role in machine learning [SL11, MJ15, LG15, JSCG16, ACB17], statistics [FCCR16, PZ16, SR04, BGKL17] and computer vision [RTG00, BvdPPH11, SdGP + 15].”
Examples are not enough, learn to criticize! Criticism for Interpretability (2016, methodology): “Lloyd and Ghahramani (2015) recently proposed an exploratory approach for statistical model criticism using the maximum mean discrepancy (MMD) two sample test, and explored the use of the witness function to identify the portions of the input space the model most misrepresents the data.”