Introduction to nonparametric estimation

Alexandre B. Tsybakov0387790527, 0387790519, 9780387790527, 9780387790510

This is a concise text developed from lecture notes and ready to be used for a course on the graduate level. The main idea is to introduce the fundamental concepts of the theory while maintaining the exposition suitable for a first approach in the field. Therefore, the results are not always given in the most general form but rather under assumptions that lead to shorter or more elegant proofs.

The book has three chapters. Chapter 1 presents basic nonparametric regression and density estimators and analyzes their properties. Chapter 2 is devoted to a detailed treatment of minimax lower bounds. Chapter 3 develops more advanced topics: Pinsker’s theorem, oracle inequalities, Stein shrinkage, and sharp minimax adaptivity.

This book will be useful for researchers and grad students interested in theoretical aspects of smoothing techniques. Many important and useful results on optimal and adaptive estimation are provided. As one of the leading mathematical statisticians working in nonparametrics, the author is an authority on the subject.

---

Table of contents :
Preface to the English Edition……Page 4
Preface to the French Edition……Page 5
Notation……Page 7
Contents……Page 9
Examples of nonparametric models and problems……Page 11
Kernel density estimators……Page 12
Mean squared error of kernel estimators……Page 14
Construction of a kernel of order……Page 20
Integrated squared risk of kernel estimators……Page 22
Lack of asymptotic optimality for fixed density……Page 26
Fourier analysis of kernel density estimators……Page 29
Unbiased risk estimation. Cross-validation density estimators……Page 37
Nonparametric regression. The Nadaraya–Watson estimator……Page 41
Local polynomial estimators……Page 44
Pointwise and integrated risk of local polynomial estimators……Page 47
Convergence in the sup-norm……Page 52
Projection estimators……Page 56
Sobolev classes and ellipsoids……Page 59
Integrated squared risk of projection estimators……Page 61
Generalizations……Page 67
Oracles……Page 69
Unbiased risk estimation for regression……Page 71
Three Gaussian models……Page 75
Notes……Page 79
Exercises……Page 82
Introduction……Page 87
A general reduction scheme……Page 89
Lower bounds based on two hypotheses……Page 91
Distances between probability measures……Page 93
Inequalities for distances……Page 96
Bounds based on distances……Page 100
Lower bounds on the risk of regression estimators at a point……Page 101
Lower bounds based on many hypotheses……Page 105
Lower bounds in L2……Page 112
Lower bounds in the sup-norm……Page 118
Fano’s lemma……Page 120
Assouad’s lemma……Page 126
The van Trees inequality……Page 130
The method of two fuzzy hypotheses……Page 135
Lower bounds for estimators of a quadratic functional……Page 138
Notes……Page 141
Exercises……Page 143
Pinsker’s theorem……Page 146
Linear minimax lemma……Page 149
Upper bound on the risk……Page 155
Lower bound on the minimax risk……Page 156
Stein’s phenomenon……Page 164
Stein’s shrinkage and the James–Stein estimator……Page 166
Other shrinkage estimators……Page 171
Superefficiency……Page 174
Unbiased estimation of the risk……Page 175
Oracle inequalities……Page 183
Minimax adaptivity……Page 188
Inadmissibility of the Pinsker estimator……Page 189
Notes……Page 194
Exercises……Page 196
Appendix……Page 200
Bibliography……Page 211
Index……Page 218