College of natural and computational science

Department of statistics
Linear regression
Biostatistics master program
Contents
1. Introduction to regression
1.1 dealing with relationship between variables (correlation and chi-square test of
association)
1.2 historical background of regression
1.3 step and objective of regression
2. Simple Linear Regression
2.1 Functional vs. Statistical Relation
2.2 Formal Statement of the Model
2.3 Graphical Representation of the Simple Linear Regression Model
2.3.1 Interpretation of the Regression Parameters
2.4 Estimation of the Regression Parameters
2.4.1 Method of Least Squares.
2.4.2 maximum likely hood estimation
2.5 Inferences in Regression Analysis
2.5.1 Inference for 1
2.5.2 Inference for 0
2.5.3 Inference for Mean Response
2.6 Predicting New Observations
2.6.1 Predicting a Single New Observation
2.6.2 Predicting the Mean of m New Observations
2.6.3 Confidence Band for the Regression Line
2.7 Analysis of Variance
2.8 F -Test versus t-Test
2.9 General Linear Test
2.10 Coefficient of Determination
3. Diagnostics and Remedial Measures
3.1.Departures from the Model
3.2.Residuals
3.3. Diagnostics Plots
3.3.1. Univariate Plots of X and Y
3.3.2. Bivariate Plots
3.4.Formal Tests
3.4.1. Tests for Normality
3.4.2. Test for Autocorrelation
3.4.3. Tests for Non-Constancy of Variance
3.4.4. Outlier Identification
3.4.5. Lack-of-Fit Test
3.5.Remedial Measures
 3.5.1. Nonlinearity
3.5.2. Non-Constancy of Error Variance
3.5.3. Outliers
3.5.4. Non-Independence
3.5.5. Non-Normality
3.6.Transformations
3.6.1. Linearizing Transforms
3.6.2. Non-Normality or Unequal Error Variance
3.6.3. Box-Cox Family of Transformations
4. Simultaneous Inference and Other Topics
4.1.Joint Estimation
4.1.1. Statement vs. Family Confidence
4.1.2. Bonferroni Joint Confidence Intervals
4.2.Regression Through the Origin
4.3.Effects of Measurement Errors
4.3.1. Errors in the Response Variable
4.3.2. Errors in the Predictor Variable
5. Multiple Linear Regression
5.1.Multiple Regression Models
5.2.General Linear Regression Model
5.2.1. General Form
5.2.2. Specific Forms
5.3.Matrix Formulation
5.3.1. Estimation of Regression Coefficients
5.3.2. Fitted Values and Residuals
5.3.3. ANOVA Results
5.3.4. F -Test for Regression Relation
5.4. Coefficients
4.4.1. Multiple Determination
4.4.2. Multiple Correlation
5.5. Inference about the Regression Parameters
5.5.1. Interval Estimation of
5.6.Inference About Mean Response
5.6.1. Interval Estimation of E(Yh)
5.6.2. Confidence Region for Regression Surface
5.6.3. Simultaneous Confidence Intervals for Several Mean Responses
5.7.Predictions
5.7.1. New Observation, Yh(new)
5.7.2. Mean of m New Observations
5.7.3. New Observations
5.8.Diagnostics and Remedial Measures
5.8.1. Diagnostic Plots
5.8.2. Formal Tests
5.8.3. Remedial Measures
5.9.Extra Sum of Squares
5.10. Coefficient of Partial Determination
 5.11. Standardized Multiple Regression
5.12. Multicolinearity
5.12.1. Uncorrelated Predictors
5.12.2. Correlated Predictors
5.12.3. Effects of Multicolinearity
6. Model Building, Diagnostics and Remedial Measures
6.1.Model Building Process
6.2.Criteria for Model Selection
6.2.1. R2p, SSEpCriterion
6.2.2. 2 , or MSEpCriterion
6.2.3. Mallows CpCriterion
6.2.4. AIC and SBC Criteria
6.2.5. P RESSpCriterion
6.3.Automatic Search Procedures for Model Selection
6.3.1. Best Subsets Algorithms
6.3.2. Stepwise Regression
6.3.3. Forward Selection
6.3.4. Backward Selection
6.3.5. Danger of Automatic Selection Procedures
6.4.Diagnostic Methods
6.4.1. Identifying Outlying Y Observations
6.4.2. Identifying Outlying X Observations
6.4.3. Identifying Influential Cases
6.4.4. Multicolinearity Diagnostics
6.5.Remedial Measures
6.5.1. Weighted Least Squares
6.5.2. Ridge Regression
6.5.3. Robust Regression
6.5.4. Regression with Auto correlated Errors
6.5.5. Nonlinear Regression
Textbook:
Montgomery, D.C., Peck, E.A. and Vining, G.G. (2002). Introduction to Linear Regression
Analysis (3rd Edition). John Wiley & Sons, Inc., New York.
References:
1. Carroll, R.J. and Ruppert, D. (1988). Transformation and Weighting in Regression.
Chapmanhall, London
2. Chatterjee, S. and Price, B. (1977). Regression Analysis by Example; Wiley, New York.
3. Draper, N.R. and Smith, H. (1998). Applied Regression Analysis (3rd Edition). John
Wiley & Sons, Inc., New York.
4. Mosteller, F. and J.W.Tukey (1977). Data Analysis and Regression: A second Course in
Statistics. Addison-Wesley, Reading, Mass.
5. Myrers, R.H. (1990). Classical and Modern Regression with Applications (2nd Edition).
PWSKent Publishers, Boston.