Distribution-Free Distribution Regression

By Barnab'as P'oczos, Aarti Singh, Alessandro Rinaldo, Larry A. Wasserman. 2013

104 citations recorded by the source corpus.

‘Distribution regression’ refers to the situation where a response Y depends on a covariate P where P is a probability distribution. The model is Y = f(P ) + where f is an unknown regression function and is a random error. Typically, we do not observe P directly, but rather, we observe a sample from P . In this paper we develop theory and methods for distribution-free versions of distribution regression. This means that we do not make strong distributional assumptions about the error term and covariate P . We prove that when the eective dimension is small enough (as measured by the doubling dimension), then the excess prediction risk converges to zero with a polynomial rate.

Subfields: Statistics and Probability, Probabilistic & Bayesian Methods

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Statistics and Probability

In Probabilistic & Bayesian Methods

Citations in the corpus

Kernel Mean Embedding of Distributions: A Review and Beyonds (2016, methodology): “More recently, modern applications of learning on distributions have also proven successful in statistical estimation and causal inference (Póczos et al. 2013, Oliva et al. 2014, Szabó et al. 2015, Lopez-Paz et al. 2015).”
Prototypal Analysis and Prototypal Regression (2017, methodology): “Póczos et al. (2013) regresses numbers from distributions through a kernel-kernel estima- tor, which involves one kernel for density estimation and another for kernel smoothing, using the distance between the distributions to weight the response variables.”
Learning Theory for Distribution Regression (2014, methodology): “…to the best of our knowledge, the only prior work addressing the consistency of regression on distributions requires kernel density estimation (Póczos et al., 2013; Oliva et al., 2014), assumes that the response variable is scalar-valued1, and the covariates are nonparametric continuous…”
Fast Distribution To Real Regression (2013, background): “DRR for the case of general input distributions in a H¨older class and a smooth class of mappings has been previously studied in [15].”
Deep Sets (2017, methodology): “Such problems are widespread, ranging from estimation of population statistics (Poczos et al., 2013), via anomaly detection in piezometer data of embankment dams (Jung et al.”
Estimating Cosmological Parameters from the Dark Matter Distribution (2016, methodology): “We analyzed the use of distribution-to-real regression (P´oczos et al., 2013) and the Double-Basis Estimator (2BE) (Oliva et al., 2014) for predicting cosmological parameters.”
From Dependence to Causation (2016, background): “In a separate strand of research, there has been multiple proposals to learn from probability distributions (Jebara et al., 2004; Hein and Bousquet, 2005; Cuturi et al., 2005; Martins et al., 2009; Muandet et al., 2012; Póczos et al., 2013).”
Operator-valued Kernels for Learning from Functional Response Data (2015, background): “There are also other papers that deal with machine learning problems where covariates are probability distributions and discuss their relation with FDA (Poczos et al., 2012, 2013; Oliva et al., 2013).”
Two-stage sampled learning theory on distributions (2014, methodology): “To the best of our knowledge, the only prior work addressing the consistency of regression on distributions requires kernel density estimation [1, 19], assumes that the response variable is scalar-valued 1 , and the covariates are nonparametric continuous distributions on R d .”
Distribution to Distribution Regression (2013, methodology): “Lemma 3 E [ I{ ∑ i Ki≤K}∑ iKi ] ≤ 1+1/KMK E [ 1 ΦP (rh) ] .”
Finding and Leveraging Structure in Learning Problems (2012, background): “Recently the papers [142, 152, 153] have considered various supervised learning problems on distributions (i.”
Linear-Time Learning on Distributions with Approximate Kernel Embeddings (2015, methodology): “The distance can then be used in kernel smoothing (Póczos et al. 2012a; Oliva, Póczos, and Schneider 2013) or Mercer kernels (Moreno, Ho, and Vasconcelos 2003; Kondor and Jebara 2003; Jebara, Kondor, and Howard 2004; Póczos et al.”