Page 163 -

162 Chapter 4 Linear Regression with One Regressor

based on these data. One way to draw the line would be to take out a pencil
and a ruler and to “eyeball” the best line you could. While this method is easy,
it is very unscientific, and different people will create different estimated
lines.

How, then, should you choose among the many possible lines? By far the
most common way is to choose the line that produces the “least squares” fit to
these data—that is, to use the ordinary least squares (OLS) estimator.

The Ordinary Least Squares Estimator

The OLS estimator chooses the regression coefficients so that the estimated

regression line is as close as possible to the observed data, where closeness is

measured by the sum of the squared mistakes made in predicting Y given X.

As discussed in Section 3.1, the sample average, Y, is the least squares estimator of

the population mean, E(Y); that is, Y minimizes the total squared estimation mistakes
n
g i= 1(Yi - m)2 among all possible estimators m [see Expression (3.2)].

The OLS estimator extends this idea to the linear regression model. Let b0 and

b1 be some estimators of b0 and b1. The regression line based on these estimators is

b0 + b1X, so the value of Yi predicted using this line is b0 + b1Xi. Thus the mistake
made in predicting the ith observation is Yi - (b0 + b1Xi) = Yi - b0 - b1Xi.

The sum of these squared prediction mistakes over all n observations is

n

ia= 1(Yi - b0 - b1Xi)2. (4.6)

The sum of the squared mistakes for the linear regression model in Expression
(4.6) is the extension of the sum of the squared mistakes for the problem of
estimating the mean in Expression (3.2). In fact, if there is no regressor, then
b1 does not enter Expression (4.6) and the two problems are identical except
for the different notation [m in Expression (3.2), b0 in Expression (4.6)]. Just
as there is a unique estimator, Y, that minimizes the Expression (3.2), so is
there a unique pair of estimators of b0 and b1 that minimize Expression (4.6).

The estimators of the intercept and slope that minimize the sum of squared
mistakes in Expression (4.6) are called the ordinary least squares (OLS) estima-
tors of b0 and b1.

OLS has its own special notation and terminology. The OLS estimator of b0
is denoted bn0, and the OLS estimator of b1 is denoted bn1. The OLS regression line,
also called the sample regression line or sample regression function, is the straight
line constructed using the OLS estimators: bn0 + bn1X. The predicted value of Yi