Page 526 -

13.2 Threats to Validity of Experiments 525

binary indicator denoting gender, then distinct causal effects for men and women
can be estimated by including the interaction variable W1i * Xi in the regression in
Equation (13.2).

Randomization based on covariates. Randomization in which the probability of
assignment to the treatment group depends on one or more observable variables
W is called randomization based on covariates. If randomization is based on
covariates, then in general the differences estimator based on Equation (13.1)
suffers from omitted variable bias. For example, Appendix 7.2 describes a hypo-
thetical experiment to estimate the causal effect of mandatory versus optional
homework in an econometrics course. In that experiment, economics majors
(Wi = 1) were assigned to the treatment group (mandatory homework, Xi = 1
with higher probability than nonmajors (Wi = 0). But if majors tend to do better
in the course than nonmajors anyway, then there is omitted variable bias because
being in the treatment group is correlated with the omitted variable, being a
major.

Because Xi is randomly assigned given Wi, this omitted variable bias can be
eliminated by using the differences estimator with the additional control variable
Wi. The random assignment of Xi given Wi (combined with the assumption of a
linear regression function) implies that, given Wi, Xi is independent of ui in
Equation (13.2). This conditional independence in turn implies conditional mean
independence, that is, E(ui ͉ Xi,Wi) = E(ui ͉ Wi) Thus the OLS estimator bn1 in
Equation (13.2) is an unbiased estimator of the causal effect when Xi is assigned
randomly based on Wi .

13.2 Threats to Validity of Experiments

Recall from Key Concept 9.1 that a statistical study is internally valid if the statis-
tical inferences about causal effects are valid for the population being studied; it
is externally valid if its inferences and conclusions can be generalized from the
population and setting studied to other populations and settings. Various real-
world problems pose threats to the internal and external validity of the statistical
analysis of actual experiments with human subjects.

Threats to Internal Validity

Threats to the internal validity of randomized controlled experiments include
failure to randomize, failure to follow the treatment protocol, attrition, experimen-
tal effects, and small sample sizes.