Design, Sites, and Data Sources
MDRC employed a cluster random assignment research design to determine whether Success for All was as effective in its larger-scale version as it had been in earlier tests of the program. In this design, 37 schools located in five districts were assigned at random: 19 to receive the program and the remaining 18 to constitute a “business-as-usual” control group.
The evaluation compared the reading outcomes of kindergartners in both sets of schools in the first year of the study, and of first-graders and second-graders in the next two years, using age-appropriate assessments that were administered individually to each child. During the second and third years of the study, students in third through fifth grades were also assessed, this time in a group setting. Student scores on state standardized tests served as additional measures of reading skills. The impact analysis also measured the program’s effects on student attendance and grade promotion.
Impacts were unlikely to occur unless schools were able to implement the program with reasonable fidelity. MDRC collected and analyzed data from a variety of sources — teacher logs, principal and teacher surveys, and rating sheets completed by the coaches — to examine the implementation experiences of the schools participating in the study. The analysis also compared the characteristics and implementation levels of the study schools with those of other schools that put Success for All in place as part of the i3 replication.