Design, Sites, and Data Sources
In Scaling Up First Things First, IRRE, the reform’s developer, provides support and technical assistance to the schools and districts into which First Things First has expanded, while MDRC acts as project manager and evaluator.
The analytic approach used for this study combines two particularly strong quasi-experimental evaluation methods: an interrupted time series analysis and a comparison schools technique. In the interrupted time series analysis, measures of student performance in schools that implemented the intervention are compared with the performance of similar students in the same schools prior to the implementation. The difference between performance levels in the two groups is referred to as a “deviation from the baseline.” A second interrupted time series analysis is conducted for a group of comparison schools in the same district that have characteristics similar to those of the intervention schools. The difference between the deviations from the baseline in the intervention schools and the deviations from the baseline in the comparison schools represents the estimated impact of the intervention.
By design, one group of expansion schools began implementing First Things First one year after the other group of schools. Thus, three years of post-implementation data are available for the earlier-starting schools and two years are available for the later-starting schools.
The expansion sites in MDRC’s study of First Things First include secondary schools in these urban, suburban, and rural school districts:
- Houston Independent School District (Houston, Texas)
- Riverview Gardens School District (suburban St. Louis County, Missouri)
- Greenville Public Schools (Greenville, Mississippi)
- Shaw Public Schools (Shaw, Mississippi)
Information about program implementation comes from annual surveys of teachers and students and from field interviews with district and school administrators, teachers, students, and others. Data sources for the impact study include school records on attendance, promotion, and test scores. A classroom observation study was also conducted to ascertain changes in instructional practices.