Local polynomial regression on unknown manifolds

By Peter J. Bickel, Bo Li. 2007

141 citations recorded by the source corpus.

We reveal the phenomenon that ``naive'' multivariate local polynomial regression can adapt to local smooth lower dimensional structure in the sense that it achieves the optimal convergence rate for nonparametric estimation of regression functions belonging to a Sobolev space when the predictor variables live on or close to a lower dimensional manifold.

Subfields: Computer Vision and Pattern Recognition, Statistics and Probability, Statistical Learning Theory

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In Statistics and Probability

In Statistical Learning Theory

Citations in the corpus

k-NN Regression Adapts to Local Intrinsic Dimension (2011, methodology): “This has recently been shown to indeed be the case for methods such as kernel regression [6], tree-based regression [7] and variants of these methods [8].”
Finding and Leveraging Structure in Learning Problems (2012, result): “Similar manifold adaptivity results have been shown in classification [54] and in non-parametric regression [30, 116].”
Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimation (2017, methodology): “(E2) β̂i is a local polynomial regression [4], d is the intrinsic dimension of the covariate space and, for every z ∈ [0, 1], f(z|x) is α times differentiable with all partial derivatives up to order α in x are bounded; (E3) β̂i is the Rodeo estimator [35], d is the number of variables that affect t”
Fast, smooth and adaptive regression in metric spaces (2009, background): “It was recently shown that certain nonparametric regressors can escape the curse of dimensionality when the intrinsic dimension of data is low ([1, 2]).”
Functional Data Analysis (2016, background): “In some cases, when data are clustered in lower-dimensional manifolds, the rates of convergence pertaining to the lower dimension will apply (Bickel & Li 2007), counteracting the “curse”.”
Unlabeled data: Now it helps, now it doesn't (2008, methodology): “If γ is ﬁxed (this is similar to ﬁxing the class of cluster-based distributions in [3] or the manifold in [4, 10]), then given enough labeled data a supervised learner can achieve optimal performance (since, eventually, it operates in regime (c) of Fig.”
Regression on manifolds: Estimation of the exterior derivative (2011, background): “We have to define the probability density with respect to a d-dimensional measure by inducing the density with the map φ(·) [8].”
Improvability Through Semi-Supervised Learning: A Survey of Theoretical Results (2019, background): “They work in a regression setting and extend work of Bickel and Li (2007) to show that there is a supervised learner that can adapt to the dimension of the manifold and thus can achieve minimax rates equivalent to a learner that directly works on the lower dimensional manifold.”