The Reading First Impact Study uses regression discontinuity analysis to assess the program’s impact on student reading achievement and classroom instructional practices. This analytic approach has a long history, and program evaluators have revived its use to measure effects when random assignment is not appropriate or feasible.

To carry out this design, the research team sought states and school districts that used explicit empirical criteria to select schools for Reading First funding. In each of the participating sites, the state or the local school districts rated candidate schools according to quantitative indicators of their need for and/or ability to benefit from the program. The schools were then ranked in order of priority for funding. Based on the resources available, all eligible schools above a pre-specified cut-off point in the ranking were slated for funding, while schools below that cut-off point were excluded from receiving Reading First grants. The impact analysis compares outcomes for schools that did or did not receive funding. Because the analysis incorporates the school selection process (represented by the rating and ranking criteria) directly into the comparison of schools, it can yield an unbiased estimate of program impacts.

The study includes more than 250 elementary schools in more than 30 school districts in 13 states. The sample consists of schools that were rated by their districts or states as eligible and appropriate for Reading First funding, as well as schools that were rated very similarly but were not able to receive a grant because of funding limitations. Within each district, between 6 and 22 schools, divided equally among Reading First and non-Reading First schools, participated in the study. The study team collected information on students and classrooms in grades 1-3 over a period of three school years.

A national reading comprehension assessment was used to measure student reading achievement. Classroom reading instruction was assessed through classroom observations and surveys of principals, reading coaches/specialists and teachers. District-level student records yielded data on attendance, mobility, and prior achievement.