本書內容分為三部分：(1)?線性回歸分析所需要的矩陣理論、多元正態分布；(2)?線性回歸的基本理論和方法，包括線性估計的一般小樣本理論、關于線性假設的?F-檢驗方法、基于線性模型的方差分析理論、變量選擇問題的討論、共線性問題、異常值問題以及?Box-Cox?模型等與線性回歸相關的內容；(3)?用于分類響應變量的Logist回歸模型的基本理論和方法。
本書要求讀者具有高等代數(或者線性代數)和概率論與數理統計的良好基礎。本書的特點之一是在盡可能少的基礎知識要求下講清線性回歸分析的理論問題，同時，本書也附帶了一些SAS代碼，這將有助于實際應用中的數據處理。
本書可供統計學專業、數學專業或者其他相關專業作為本科生回歸分析課程教材使用，也可作為非統計學專業的研究人員學習回歸分析基礎理論的參考書使用。
本書內容能夠分為三部分：（1）線性回歸分析所需要的矩陣代數理論、多元正態分布；（2）線性回歸的基本利率和方法，包括線性估計的一般小樣本理論、關于線性線性假設的F-檢驗方法、基于線性模型的方差分析理論、變量選擇問題的討論、共線性問題、異常值問題以及Box-Cox模型等與線性回歸相關的內容；（3）用于分類響應變量的Logist回歸模型的基本理論和方法。本書要求讀者具有高等代數（或者線性代數）和概率論與數理統計的良好基礎。本書的特點之一是在盡可能少的基礎知識要求下講清回歸分析的理論問題，同事，本書也附帶了一些SAS代碼，這將有助于實際應用中的數據處理。本書可供統計學專業、數學專業或者其他相關專業作為本科生回歸分析課程教材使用，也可作為非統計學專業的研究人員學習回歸分析基礎理論的參考使用。
吳賢毅，華東師范大學金融與統計學院教授，博士生導師，研究領域包括隨機調度，概率統計，非壽險精算，在隨機調度，概率統計以及非壽險精算的國際主流雜志發表過數十篇學術研究論文，在隨機調度方面的研究獲得過三次國家自然科學基金資助，其在隨機調度方面的研究成果發表于Operations Research,European Journal of Operations Research,Journal of Scheduling 等。
Contents
Chapter 1 Preliminaries: Matrix Algebra and Random Vectors ........ 1
1.1 Preliminary matrix algebra ............................................................ 1
1.1.1 Trace and eigenvalues.......................................................... 1
1.1.2 Symmetric matrices............................................................. 3
1.1.3 Idempotent matrices and orthogonal projection..................... 6
1.1.4 Singular value decomposition ............................................... 9
1.1.5 Vector di.erentiation and generalized inverse .......................10
1.1.6 Exercises ...........................................................................10
1.2 Expectation and covariance...........................................................11
1.2.1 Basic properties .................................................................11
1.2.2 Mean and variance of quadratic forms .................................12
1.2.3 Exercises ...........................................................................14
1.3 Moment generating functions and independence .............................16
1.3.1 Exercises ...........................................................................17
Chapter 2 Multivariate Normal Distributions.....................................18
2.1 De.nitions and fundamental results ...............................................18
2.2 Distribution of quadratic forms .....................................................25
2.3 Exercises......................................................................................27
Chapter 3 Linear Regression Models ...................................................29
3.1 Introduction.................................................................................29
3.2 Regression interpreted as conditional mean ....................................31
3.3 Linear regression interpreted as linear prediction ............................33
3.4 Some nonlinear regressions............................................................34
3.5 Typical data structure of linear regression models ..........................35
3.6 Exercises......................................................................................36
Chapter 4 Estimation and Distribution Theory ..................................38
4.1 Least squares estimation (LSE) .....................................................38
4.1.1 Motivation: why is LS reasonable........................................38
Regression Analysis
4.1.2 The LS solution .................................................................40
4.1.3 Exercises ...........................................................................48
4.2 Properties of LSE .........................................................................50
4.2.1 Small sample distribution-free properties .............................51
4.2.2 Properties under normally distributed errors........................55
4.2.3 Asymptotic properties ........................................................57
4.2.4 Exercises ...........................................................................60
4.3 Estimation under linear restrictions ...............................................60
4.4 Introducing further explanatory variables and related topics ...........67
4.4.1 Introducing further explanatory variables ............................67
4.4.2 Centering and scaling the explanatory variables ...................71
4.4.3 Estimation in terms of linear prediction...............................72
4.4.4 Exercises ...........................................................................73
4.5 Design matrices of less than full rank.............................................74
4.5.1 An example .......................................................................74
4.5.2 Estimability.......................................................................74
4.5.3 Identi.ability under Constraints..........................................76
4.5.4 Dropping variables to change the model ..............................77
4.5.5 Exercises ...........................................................................77
4.6 Generalized least squares ..............................................................78
4.6.1 Basic theory ......................................................................78
4.6.2 Incorrect speci.cation of variance matrix.............................80
4.6.3 Exercises ...........................................................................83
4.7 Bayesian estimation......................................................................83
4.7.1 The basic idea....................................................................83
4.7.2 Normal-noninformative structure ........................................84
4.7.3 Conjugate priors ................................................................85
4.8 Numerical examples......................................................................86
4.9 Exercises......................................................................................90
Chapter 5 Testing Linear Hypotheses ..................................................92
5.1 Linear hypotheses.........................................................................92
5.2 F -Test .........................................................................................93
5.2.1 F -test................................................................................94
Contents VII
5.2.2 What are actually tested ....................................................95
5.2.3 Examples...........................................................................96
5.3 Con.dence ellipse ....................................................................... 101
5.4 Prediction and calibration...........................................................103
5.5 Multiple correlation coe.cient .................................................... 105
5.5.1 Variable selection ............................................................. 105
5.5.2 Multiple correlation coe.cient: straight line ...................... 106
5.5.3 Multiple correlation coe.cient: multiple regression ............ 108
5.5.4 Partial correlation coe.cient ............................................ 110
5.5.5 Adjusted multiple correlation coe.cient ............................ 111
5.6 Testing linearity: goodness-of-.t test ........................................... 112
5.7 Multiple comparisons..................................................................114
5.7.1 Simultaneous inference ..................................................... 114
5.7.2 Some classical methods for simultaneous intervals .............. 116
5.8 Univariate analysis of variance .................................................... 120
5.8.1 ANOVA model................................................................. 120
5.8.2 ANCOVA model .............................................................. 126
5.8.3 SAS procedures for ANOVA ............................................. 127
5.9 Exercises.................................................................................... 129
Chapter 6 Variable Selection............................................................... 133
......
新書資訊