Page 753 -

752 Chapter 18 The Theory of Multiple Regression

The next two sections turn to the theory of efficient estimation of the coefficients
of the multiple regression model. Section 18.5 generalizes the Gauss–Markov theorem
to multiple regression. Section 18.6 develops the method of generalized least
squares (GLS).

The final section takes up IV estimation in the general IV regression model
when the instruments are valid and strong. This section derives the asymptotic
distribution of the TSLS estimator when the errors are heteroskedastic and provides
expressions for the standard error of the TSLS estimator. The TSLS estimator is one
of many possible GMM estimators, and this section provides an introduction to
GMM estimation in the linear IV regression model. It is shown that the TSLS estimator
is the efficient GMM estimator if the errors are homoskedastic.

Mathematical prerequisite. The treatment of the linear model in this chapter uses
matrix notation and the basic tools of linear algebra and assumes that the reader
has taken an introductory course in linear algebra. Appendix 18.1 reviews vectors,
matrices, and the matrix operations used in this chapter. In addition, multivariate
calculus is used in Section 18.1 to derive the OLS estimator.

18.1 The Linear Multiple Regression Model
and OLS Estimator in Matrix Form

The linear multiple regression model and the OLS estimator can each be repre-
sented compactly using matrix notation.

The Multiple Regression Model in Matrix Notation

The population multiple regression model (Key Concept 6.2) is

Yi = b0 + b1X1i + b2X2i + g + bkXki + ui, i = 1, c, n. (18.1)

To write the multiple regression model in matrix form, define the following vectors
and matrices:

Y1 u1 1 X11 g Xk1 X1′ b0

Y = ±fY2 ≤ , U = ±fu2 ≤ , X = 1 X12 g Xk2 ≤ = X 2′ ≤ , and B = ±fb1 ≤ , (18.2)
±f f f f ±f

Yn un 1 X1n g Xkn X′n bk