Linear and convex aggregation of density estimators

By Philippe Rigollet, Alexandre B. Tsybakov. 2006

142 citations recorded by the source corpus.

We study the problem of finding the best linear and convex combination of M estimators of a density with respect to the mean squared risk. We suggest aggregation procedures and we prove sharp oracle inequalities for their risks, i.e., oracle inequalities with leading constant 1. We also obtain lower bounds showing that these procedures attain optimal rates of aggregation. As an example, we consider aggregation of multivariate kernel density estimators with different bandwidths. We show that linear and convex aggregates mimic the kernel oracles in asymptotically exact sense. We prove that, for Pinsker’s kernel, the proposed aggregates are sharp asymptotically minimax simultaneously over a large scale of Sobolev classes of densities. Finally, we provide simulations demonstrating performance of the convex aggregation procedure.

Subfields: Epidemiology, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/39468b3d8192a32094222cd70661df614bf66bfa

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Epidemiology

In Statistics and Probability

Citations in the corpus

Bandwidth selection in kernel density estimation: Oracle inequalities and adaptive minimax optimality (2010, methodology): “Kernel density estimates originate in Rosenblatt (1956) and Parzen (1962); this is one of the most popular techniques for estimating densities [Silverman (1986), Devroye and Györfi (1985)].”
On adaptive minimax density estimation on $$R^d$$Rd (2012, background): “2001), Efroimovich (1986,2008), Hasminskii and Ibragimov (1990), Donoho et al. (1996), Golubev(1992),Kerkyacharian, Picard and Tribouley(1996), Rigollet (2006), Massart (2007)[Chapter 7], Samarov and Tsybakov (2007), Rigollet and Tsybakov (2007) and Birg´e (2008). It is not our aim here to provide a”
Semiparametric Counterfactual Density Estimation (2021, methodology): “One could alternatively take q(y) = 0 and let bj(y) be (the linear span of) a collection of d candidate densities, in which case the above could be viewed as a linear aggregation [Rigollet and Tsybakov, 2007].”
SPADES AND MIXTURE MODELS (2009, background): “ and even more pointed (without extra logarithmic factors in the rate and sometimes with exact asymptotic minimax constants) are known for various other adaptive density estimators, see,for instance, [16, 6, 19, 21, 28, 29] and the references cited therein. These papers consider classes of densities”