Page 797 -

796 Chapter 18 The Theory of Multiple Regression

The Mean Vector and Covariance Matrix

The first and second moments of an m * 1 vector of random variables, V =
(V1 V2 g Vm)′, are summarized by its mean vector and covariance matrix.

Because V is a vector, the vector of its means—that is, its mean vector—is E(V) = MV.
The ith element of the mean vector is the mean of the ith element of V.

The covariance matrix of V is the matrix consisting of the variance var(Vi), i = 1, . . . , m,
along the diagonal and the (i, j) off-diagonal elements cov(Vi, Vj). In matrix form, the
covariance matrix V is

var(V1) g cov(V1, Vm)
f f S . (18.72)
V = E[(V - MV)(V - MV)′] = C f

cov(Vm, V1) g var(Vm)

The Multivariate Normal Distribution

The m * 1 vector random variable V has a multivariate normal distribution with mean
vector MV and covariance matrix V if it has the joint probability density function

f(V ) = 1 V) exp c - 1 (V - MV)′ V-1(V - MV) d ,(18.73)
2(2p)mdet( 2

where det( V) is the determinant of the matrix V. The multivariate normal distribution
is denoted N(MV, V).

An important fact about the multivariate normal distribution is that if two jointly

normally distributed random variables are uncorrelated (equivalently, have a block-diagonal

covariance matrix), then they are independently distributed. That is, let V1 and V2 be
jointly normally distributed random variables with respective dimensions m1 * 1 and
m2 * 1. Then if cov(V1, V2) = E[(V1 - MV1)(V2 - MV2)′] = 0m1 * m2, V1 and V2 are
independent.

If {Vi} are i.i.d. N(0, sv2), then V = sv2 Im, and the multivariate normal distribution
simplifies to the product of m univariate normal densities.

Distributions of Linear Combinations and Quadratic
Forms of Normal Random Variables

Linear combinations of multivariate normal random variables are themselves normally
distributed, and certain quadratic forms of multivariate normal random variables have a
chi-squared distribution. Let V be an m * 1 random variable distributed N(MV, V), let A