Page 527 -

526 Chapter 13 Experiments and Quasi-Experiments

Failure to randomize. If the treatment is not assigned randomly, but instead is
based in part on the characteristics or preferences of the subject, then experimen-
tal outcomes will reflect both the effect of the treatment and the effect of the
nonrandom assignment. For example, suppose that participants in a job training
program experiment are assigned to the treatment group depending on whether
their last name falls in the first or second half of the alphabet. Because of ethnic
differences in last names, ethnicity could differ systematically between the treat-
ment and control groups. To the extent that work experience, education, and
other labor market characteristics differ by ethnicity, there could be systematic
differences between the treatment and control groups in these omitted factors
that affect outcomes. In general, nonrandom assignment can lead to correlation
between Xi and ui in Equations (13.1) and (13.2), which in turn leads to bias in the
estimator of the treatment effect.

It is possible to test for randomization. If treatment is randomly received,
then Xi will be uncorrelated with observable pretreatment individual characteris-
tics W. Thus, a test for random receipt of treatment entails testing the hypothesis
that the coefficients on W1i, c, Wri are zero in a regression of Xi on W1i, c, Wri.
In the job training program example, regressing receipt of job training (Xi) on
gender, race, and prior education (W’s), and then computing the F-statistic testing
whether the coefficients on the W’s are zero, provides a test of the null hypothesis
that treatment was randomly received, against the alternative hypothesis that
receipt of treatment depends on gender, race, or prior education. If the experi-
mental design performs randomization conditional on covariates, then those
covariates would be included in the regression and the F-test would test the coef-
ficients on the remaining W’s.1

Failure to follow the treatment protocol. In an actual experiment, people do not
always do what they are told. In a job training program experiment, for example,
some of the subjects assigned to the treatment group might not show up for the
training sessions and thus not receive the treatment. Similarly, subjects assigned
to the control group might somehow receive the training anyway, perhaps by
making a special request to an instructor or administrator.

The failure of individuals to follow completely the randomized treatment pro-
tocol is called partial compliance with the treatment protocol. In some cases, the

1In this example, Xi is binary, so, as discussed in Chapter 11, the regression of Xi on W1i, c, Wri is a
linear probability model and heteroskedasticity-robust standard errors are essential. Another way to
test the hypothesis that E(Xi ͉W1i, c, Wri) does not depend on W1i, c, Wr i when Xi is binary is to
use a probit or logit model (see Section 11.2).