Praise for the Fourth Edition: "This book is ...an excellent source of examples for regression analysis. It has been and still is readily readable and understandable." --Journal of the American Statistical Association Regression analysis is a conceptually simple method for investigating relationships among variables. Carrying out a successful application of regression analysis, however, requires a balance of theoretical results, empirical rules, and subjective judgment. Regression Analysis by Example, Fifth Edition has been expanded and thoroughly updated to reflect recent advances in the field. The emphasis continues to be on exploratory data analysis rather than statistical theory. The book offers in-depth treatment of regression diagnostics, transformation, multicollinearity, logistic regression, and robust regression. The book now includes a new chapter on the detection and correction of multicollinearity, while also showcasing the use of the discussed methods on newly added data sets from the fields of engineering, medicine, and business. The Fifth Edition also explores additional topics, including: Surrogate ridge regression Fitting nonlinear models Errors in variables ANOVA for designed experiments Methods of regression analysis are clearly demonstrated, and examples containing the types of irregularities commonly encountered in the real world are provided. Each example isolates one or two techniques and features detailed discussions, the required assumptions, and the evaluated success of each technique. Additionally, methods described throughout the book can be carried out with most of the currently available statistical software packages, such as the software package R. Regression Analysis by Example, Fifth Edition is suitable for anyone with an understanding of elementary statistics.
Preface xiv 1 Introduction 1 1.1 What Is Regression Analysis? 1 1.2 Publicly Available Data Sets 2 1.3 Selected Applications of Regression Analysis 3 1.4 Steps in Regression Analysis 13 1.5 Scope and Organization of the Book 21 Exercises 23 2 Simple Linear Regression 25 2.1 Introduction 25 2.2 Covariance and Correlation Coefficient 25 2.3 Example: Computer Repair Data 30 2.4 The Simple Linear Regression Model 32 2.5 Parameter Estimation 33 2.6 Tests of Hypotheses 36 2.7 Confidence Intervals 41 2.8 Predictions 41 2.9 Measuring the Quality of Fit 43 2.10 Regression Line Through the Origin 46 2.11 Trivial Regression Models 48 2.12 Bibliographic Notes 49 Exercises 49 3 Multiple Linear Regression 57 3.1 Introduction 57 3.2 Description of the Data and Model 57 3.3 Example: Supervisor Performance Data 58 3.4 Parameter Estimation 61 3.5 Interpretations of Regression Coefficients 62 3.6 Centering and Scaling 64 3.7 Properties of the Least Squares Estimators 67 3.8 Multiple Correlation Coefficient 68 3.9 Inference for Individual Regression Coefficients 69 3.10 Tests of Hypotheses in a Linear Model 71 3.11 Predictions 81 3.12 Summary 82 Exercises 82 Appendix: Multiple Regression in Matrix Notation 89 4 Regression Diagnostics: Detection of Model Violations 93 4.1 Introduction 93 4.2 The Standard Regression Assumptions 94 4.3 Various Types of Residuals 96 4.4 Graphical Methods 98 4.5 Graphs Before Fitting a Model 101 4.6 Graphs After Fitting a Model 105 4.7 Checking Linearity and Normality Assumptions 105 4.8 Leverage, Influence, and Outliers 106 4.9 Measures of Influence 111 4.10 The Potential-Residual Plot 115 4.11 What to Do with the Outliers? 116 4.12 Role of Variables in a Regression Equation 117 4.13 Effects of an Additional Predictor 122 4.14 Robust Regression 123 Exercises 123 5 Qualitative Variables as Predictors 129 5.1 Introduction 129 5.2 Salary Survey Data 130 5.3 Interaction Variables 133 5.4 Systems of Regression Equations 136 5.5 Other Applications of Indicator Variables 147 5.6 Seasonality 148 5.7 Stability of Regression Parameters Over Time 149 Exercises 151 6 Transformation of Variables 163 6.1 Introduction 163 6.2 Transformations to Achieve Linearity 165 6.3 Bacteria Deaths Due to XRay Radiation 167 6.4 Transformations to Stabilize Variance 171 6.5 Detection of Heteroscedastic Errors 176 6.6 Removal of Heteroscedasticity 178 6.7 Weighted Least Squares 179 6.8 Logarithmic Transformation of Data 180 6.9 Power Transformation 181 6.10 Summary 185 Exercises 186 7 Weighted Least Squares 191 7.1 Introduction 191 7.2 Heteroscedastic Models 192 7.3 Two-Stage Estimation 195 7.4 Education Expenditure Data 197 7.5 Fitting a Dose-Response Relationship Curve 206 Exercises 208 8 The Problem of Correlated Errors 209 8.1 Introduction: Autocorrelation 209 8.2 Consumer Expenditure and Money Stock 210 8.3 Durbin-Watson Statistic 212 8.4 Removal of Autocorrelation by Transformation 214 8.5 Iterative Estimation With Autocorrelated Errors 216 8.6 Autocorrelation and Missing Variables 217 8.7 Analysis of Housing Starts 218 8.8 Limitations of Durbin-Watson Statistic 222 8.9 Indicator Variables to Remove Seasonality 223 8.10 Regressing Two Time Series 226 Exercises 228 9 Analysis of Collinear Data 233 9.1 Introduction 233 9.2 Effects of Collinearity on Inference 234 9.3 Effects of Collinearity on Forecasting 240 9.4 Detection of Collinearity 245 Exercises 254 10 Working With Collinear Data 259 10.1 Introduction 259 10.2 Principal Components 259 10.3 Computations Using Principal Components 263 10.4 Imposing Constraints 263 10.5 Searching for Linear Functions of the beta 's 267 10.6 Biased Estimation of Regression Coefficients 272 10.7 Principal Components Regression 272 10.8 Reduction of Collinearity in the Estimation Data 274 10.9 Constraints on the Regression Coefficients 276 10.10 Principal Components Regression: A Caution 277 10.11 Ridge Regression 280 10.12 Est
