Cluster Random Assignment

If random assignment is the “gold standard” of program evaluation, what does a researcher do when the focus of an intervention is a group, rather than individuals — making assignment of individual participants to program and control groups impossible? For example, the groups may be organizations, like schools or hospitals or businesses, or they may be geographically defined, like neighborhoods or even cities.

One answer is cluster random assignment — in which groups of people are assigned randomly to a program or control situation. MDRC has accumulated a wealth of experience and knowledge in the design and analysis of cluster random assignment experiments. Howard Bloom has authored and co-authored with colleagues inside and outside of MDRC a number of publications about cluster randomized trials that are read by graduate students at major universities and by researchers in public, private, and nonprofit organizations, and that are cited as recommended reading in requests for proposals (RFPs) released by foundations and federal agencies.

Until relatively recently, it was common for studies that used cluster random assignment to analyze the resulting data as if individuals rather than groups had been randomly assigned — a procedure that led to erroneous conclusions. But because individuals are “nested” within the unit of randomization — the group or cluster — at least two sources of sampling error enter the picture: one resulting from variation in outcomes among groups in the treatment and control samples and one resulting from variation in outcomes among individuals within each group. Unless both kinds of sampling error are included in the standard error, investigators may wrongly decide that a program is making a significant difference when, in fact, it is not. The strategy that Bloom and other leading social scientists employ in cluster randomized trials — referred to as “multilevel modeling” or as “hierarchical modeling” — takes account of both sources of sampling error in producing impact estimates.

MDRC has used cluster random assignment in a variety of settings. Our evaluation of Achieve, an employer-based program for reducing job turnover rates among low-wage workers in the health care industry, 44 health care firms in Cleveland that volunteered to participate in the study were randomly assigned to program and control groups. Schools provide especially opportune settings for cluster random assignment because many educational reforms aim to change the culture and practices of the school as a whole, or to affect the learning of students at the classroom level. MDRC and colleagues in other organizations have undertaken several evaluations, funded by the U.S. Department of Education, that called for school-level randomization, including the Middle School Mathematics Professional Development Impact Study,  and the Reading Professional Development Impact Study.

Community colleges, too, have begun to sign on for cluster random assignment studies. In MDRC’s evaluation of the South Texas College Beacon Mentoring Program, 83 sections of developmental (remedial) math or college algebra were randomly assigned either to receive a mentor to be part of the control group.

Cluster random assignment has become a valuable and frequently-used tool in the program evaluation toolkit. MDRC is proud to have played a role in building appreciation and understanding of this methodology throughout the evaluation community.