On risk bounds in isotonic and other shape restricted regression problems

By Sabyasachi Chatterjee, Adityanand Guntuboyina, Bodhisattva Sen. 2013

143 citations recorded by the source corpus.

We consider the problem of estimating an unknown $\theta\in {\mathbb{R}}^n$ from noisy observations under the constraint that $\theta$ belongs to certain convex polyhedral cones in ${\mathbb{R}}^n$. Under this setting, we prove bounds for the risk of the least squares estimator (LSE). The obtained risk bound behaves differently depending on the true sequence $\theta$ which highlights the adaptive behavior of $\theta$. As special cases of our general result, we derive risk bounds for the LSE in univariate isotonic and convex regression. We study the risk bound in isotonic regression in greater detail: we show that the isotonic LSE converges at a whole range of rates from $\log n/n$ (when $\theta$ is constant) to $n^{-2/3}$ (when $\theta$ is uniformly increasing in a certain sense). We argue that the bound presents a benchmark for the risk of any estimator in isotonic regression by proving nonasymptotic local minimax lower bounds. We prove an analogue of our bound for model misspecification where the true $\theta$ is not necessarily nondecreasing.

Subfields: Computational Mechanics, Statistics and Probability

View Paper: https://www.semanticscholar.org/paper/06633d9ac236a1e0a708d431390070027a410795

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computational Mechanics

In Statistics and Probability

Citations in the corpus

Nonparametric Shape-Restricted Regression (2017, background): “This inequality has been proved by [19, Section 4] improving earlier results of [28, 67] which had superfluous multiplicative constants.”
Optimal rates of statistical seriation (2016, methodology): “In this context, [20] showed that the LS estimator satisfies the adaptive bound”
On matrix estimation under monotonicity constraints (2015, background): “[7] where inequality (3) was proved with an additional constant multiplicative factor.”
Isotonic regression in general dimensions (2017, background): “In this subsection, we consider the adaptation behaviour of the least squares estimator in dimensions d ≥ 2 (again, the d = 2 case is covered in Chatterjee, Guntuboyina and Sen (2017)).”
Least squares estimation in the monotone single index model (2016, methodology): “Following Chatterjee et al. (2015), we study the global convergence rate when the truth Ψ0 is piecewise constant.”
Sharp oracle inequalities for Least Squares estimators in shape restricted regression (2015, methodology): “2, these oracle inequalities hold in deviation with exponential probability bounds and in expectation for any μ ∈ R, whereas previously known oracle inequalities from [11, 7] only hold in expectation under the additional assumption that μ ∈ KC x1,.”
Bayesian fractional posteriors (2016, background): “Sharp oracle inequalities for isotonic regression for d = 1 has been recently established in [8], improving on a previous risk bound by [13].”