Preface xi
1 Introduction 1
1.1 Multivariate analysis in a nutshell 2
1.2 The path that lies ahead 13
2 Vector and function spaces 15
2.1 Metric spaces 16
2.2 Vector and normed spaces 20
2.3 Banach and Lp spaces 26
2.4 Inner Product and Hilbert spaces 31
2.5 The projection theorem and orthogonal decomposition 38
2.6 Vector integrals 40
2.7 Reproducing kernel Hilbert spaces 46
2.8 Sobolev spaces 55
3 Linear operator and functionals 61
3.1 Operators 62
3.2 Linear functionals 66
3.3 Adjoint operator 71
3.4 Nonnegative, square-root, and projection operators 74
3.5 Operator inverses 77
3.6 Frechet and Gateaux derivatives 83
3.7 Generalized Gram Schmidt decompositions 87
4 Compact operators and singular value decomposition 91
4.1 Compact operators 92
4.2 Eigenvalues of compact operators 96
4.3 The singular value decomposition 103
4.4 Hilbert Schmidt operators 107
4.5 Trace class operators 113
4.6 Integral operators and Mercer s Theorem 116
4.7 Operators on an RKHS 123
4.8 Simultaneous diagonalization of two nonnegative definite operators 126
5 Perturbation theory 129
5.1 Perturbation of self-adjoint compact operators 129
5.2 Perturbation of general compact operators 140
6 Smoothing and regularization 147
6.1 Functional linear model 147
6.2 Penalized least squares estimators 150
6.3 Bias and variance 157
6.4 A computational formula 158
6.5 Regularization parameter selection 161
6.6 Splines 165
7 Random elements in a Hilbert space 175
7.1 Probability measures on a Hilbert space 176
7.2 Mean and covariance of a random element of a Hilbert space 178
7.3 Mean-square continuous processes and the Karhunen Loeve Theorem 184
7.4 Mean-square continuous processes in L2 (E,B(E), mu) 190
7.5 RKHS valued processes 195
7.6 The closed span of a process 198
7.7 Large sample theory 203
8 Mean and covariance estimation 211
8.1 Sample mean and covariance operator 212
8.2 Local linear estimation 214
8.3 Penalized least-squares estimation 231
9 Principal components analysis 251
9.1 Estimation via the sample covariance operator 253
9.2 Estimation via local linear smoothing 255
9.3 Estimation via penalized least squares 261
10 Canonical correlation analysis 265
10.1 CCA for random elements of a Hilbert space 267
10.2 Estimation 274
10.3 Prediction and regression 281
10.4 Factor analysis 284
10.5 MANOVA and discriminant analysis 288
10.6 Orthogonal subspaces and partial cca 294
11 Regression 305
11.1 A functional regression model 305
11.2 Asymptotic theory 308
11.3 Minimax optimality 318
11.4 Discretely sampled data 321
References 327
Index 331
Notation Index 334
