A partition-based similarity for classification distributions

By Hayden S. Helm, Ronak D. Mehta, Brandon Duderstadt, Weiwei Yang, Christopher M. White, Ali Geisa, Joshua T. Vogelstein, Carey E. Priebe. arXiv.org, 2020

6 citations recorded by the source corpus.

Herein we define a measure of similarity between classification distributions that is both principled from the perspective of statistical pattern recognition and useful from the perspective of machine learning practitioners. In particular, we propose a novel similarity on classification distributions, dubbed task similarity, that quantifies how an optimally-transformed optimal representation for a source distribution performs when applied to inference related to a target distribution. The definition of task similarity allows for natural definitions of adversarial and orthogonal distributions. We highlight limiting properties of representations induced by (universally) consistent decision rules and demonstrate in simulation that an empirical estimate of task similarity is a function of the decision rule deployed for inference. We demonstrate that for a given target distribution, both transfer efficiency and semantic similarity of candidate source distributions correlate with empirical task similarity.

Subfields: Computer Vision and Pattern Recognition, Statistics and Probability

View Paper: https://arxiv.org/abs/2011.06557

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistics and Probability

Citations in the corpus

Statistical inference on black-box generative models in the data kernel perspective space (2024, unknown): “…generally without information loss, it may be necessary to replace the Frobenius norm of the differences of the average embedded response with a more expressive distance such as one defined directly on the cumulative distributions of responses or with a task-specific distance (Helm et al., 2020).”
A Statistical Turing Test for Generative Models (2023, unknown): “The black-box setting is extremely limiting in terms of feasible types of model comparisons but is along a similar direction as previous work on similarity measures for partition based classification rules [17].”
Approximately optimal domain adaptation with Fisher's Linear Discriminant Analysis (2023, methodology): “Instead, we approximate E ( h α ) with ˆ E ( h α ) , which we can evaluate via Monte Carlo integration by sampling from the distribution of ω α (derived in Section 3.3) and using the plug-in estimates for ν (0) and Σ (0) .”
Mental State Classification Using Multi-Graph Features (2022, background): “[37] and successful techniques will likely be of unusual importance in biosensing settings to keep on-device models as light, private, and effective as possible.”
Inducing a hierarchy for multi-class classification problems (2021, methodology): “Herein we study task similarity (Helm et al., 2020) as a similarity used to induce a hierarchy.”