| Elementary schools are expected to teach children many skills that form
the basis for success in later education and the labor market. Schools’
typical response to low academic achievement, especially among poor and
minority students, has been remedial education programs that slow down
the pace of instruction or simplify the content of the curriculum. Past
research suggests that this approach has serious limitations and can put
students who are at risk of school failure at still greater disadvantage.
The Accelerated Schools approach charts a different course,
calling for a complex series of school changes designed to “accelerate”
the learning of at-risk children. Developed over the last 15 years by
Dr. Henry M. Levin and his colleagues under the aegis of the Accelerated
Schools Project (ASP), the approach is now in use in more than 1,000 elementary
and middle schools. Accelerated Schools aim to (1) create a new, supportive
school culture that helps all children learn and sets high expectations
for teachers and students; (2) institute a new governance structure consisting
of a structured process for taking stock of the school’s problems
and strengths, work groups (called cadres) devoted to school issues of
concern, and school-wide meetings at which important decisions are reached
by consensus; (3) use powerful learning, a teaching approach that is more
challenging, interactive, project-based, and relevant for students than
traditional instruction; and (4) receive technical assistance in implementing
the reform from ASP’s National Center and 11 Satellite Centers across
the country.
At ASP’s invitation, the
Manpower Demonstration Research Corporation (MDRC) conducted an independent
evaluation of Accelerated Schools at the elementary level. The objective
of the study, which was funded by the Ford Foundation, was to assess whether
the Accelerated Schools approach improved student achievement in a small
sample of schools that served at-risk students and had launched the reform
early in its development. This report, the culmination of the study, presents
findings on the reform’s implementation and impacts. All eight schools
in the study adopted the reform in the early 1990s, before powerful learning
— a key component — was refined to make its implications for
classroom practice clearer and before the technical assistance provided
by ASP was extended and improved. The study examined the Accelerated Schools
model’s effects on the reading and math achievement test scores of
the schools’ third-grade cohorts during each of the five years after
the reform began to be implemented. The third grade was chosen as the
focus of the study because it marks a critical point in the development
of basic reading and math skills and is late enough in elementary school
for students to have been exposed to the reform for a considerable time
(up to three years).
I.
Findings in Brief
Three years after launching the
Accelerated Schools model, all the study schools had adopted the practices
aimed at improving school culture and governance. Changes in curriculum
and instruction were implemented less systematically and more slowly,
with many schools starting to address these topics only in the third or
fourth year of implementation.
- At the end of the five-year follow-up period, the average third-grade
math and reading test scores in the participating schools were higher
than those during the baseline period by a statistically significant
amount. The magnitude of these test score improvements is similar to
that found in the well-known Tennessee class-size experiment.
- The findings suggest that students who would have been in the middle
of their school’s test score distribution without the reform were
the most likely to experience an improvement in test scores.
- The observed increases in average test scores were largest among
the study schools that had the lowest average test scores before implementation
of the Accelerated Schools model.
- Improving test scores in schools that serve at-risk students has
proven to be an extremely difficult challenge to meet, making the present
findings of particular interest. Nevertheless, the effects found here
should be interpreted with caution because they are based on a sample
of only eight Accelerated Schools that had reached an advanced stage
of implementation by the early 1990s, did not emerge until four to five
years after the reform was launched, and may not persist in later grades.
II.
Study Design
To estimate the reform’s impacts on student achievement over a
period of sufficient length, the schools in the study were required
to have launched the Accelerated Schools reform at least five years
before the start of the study period and to have five years of baseline
test score data. ASP staff nominated 91 elementary schools that they
believed had been implementing key elements of the reform for at least
five years, and the MDRC research team independently selected schools
from this group. The eight urban and rural schools in the final sample
were chosen for serving mostly at-risk students, having launched the
reform and implemented its main components by the early 1990s, not having
instituted other major reforms during the study period (to help rule
out alternative explanations for any test score impacts observed), and
being able to supply the requisite data. It is important to bear in
mind that the study schools, though not a hand-picked group selected
on the basis of test score trends, are also not a representative sample
of Accelerated Schools from the early 1990s.
The study period encompassed a baseline (prelaunch) period of three
years [1] and a follow-up
(postlaunch) period of five years. The findings are based on an interrupted
time-series design in which the test score level in each subject
(reading and math) that would be expected in the absence of the Accelerated
Schools reform was estimated using test score data from the baseline
period. Specifically, it was assumed that, if the reform had not been
implemented, the test score level during the follow-up period would
have been the same as the average level during the three baseline years.
The difference between the expected level and the actual level in each
follow-up year was taken as an estimate of the Accelerated Schools reform’s
impact on test scores in that year.
[2] The impact analyses focus on the entire sample of schools
rather than on individual schools.
III.
Implementation of the Accelerated Schools Approach
- During their first three years of implementation, the study schools
focused on reforming school governance and culture; substantial changes
in curriculum and instruction were typically not made until the third
or fourth year.
Most schools used the reform’s “inquiry” process to
take stock of current school performance and to identify key issues
to address. They then used cadres to analyze courses of action and to
develop recommendations, a steering committee to advise the cadres and
coordinate the process, and school-wide meetings to decide important
questions. The adoption of these processes was generally credited with
creating an atmosphere of greater trust and support for staff, raising
expectations of teachers and students, and increasing staff participation
in decision-making.
- Administrators and faculty working in these schools reported that
when they first adopted the Accelerated Schools model, powerful learning
was not precisely or concretely defined.
Many staff were confused about the instructional changes prescribed
by the Accelerated Schools model. They also reported a lack of explicit
guidance from ASP, especially about instructional practices, after the
start-up phase of the reform. During the first three years of the follow-up
period, most of the schools used the governance procedures to reach
their own decisions about instruction, with an emphasis on aligning
the curriculum with new state or local standards rather than on developing
new instructional techniques or new classroom practices.
IV.
Impacts on Student Achievement
- Impacts tracked implementation.
On average, there was no systematic change in test scores during the
first two years of Accelerated Schools implementation, during which
time the schools focused on establishing supportive cultures and decision-making
processes. During the third year, test scores declined somewhat, as
schools began to make instructional changes that may have temporarily
disrupted instruction. During the last two years of the follow-up period,
test scores gradually rose.
- In the fifth follow-up year, the average test scores in the sample
schools exceeded the baseline level in both reading and math by a statistically
significant amount.
In the fifth year of the
follow-up period, the average third-grade reading and math test scores
in the study schools overall were 0.19 standard deviation and 0.24 standard
deviation above their respective baseline averages. These differences,
which are statistically significant, are small to modest by the conventional
standards of evaluation research. However, the impacts found in the Tennessee
class-size experiment — which are widely considered policy-relevant
— are of similar magnitude. The positive impacts in the last follow-up
year are especially noteworthy given the low cost of this early version
of the Accelerated Schools model relative to that of other school reform
approaches.
- The average reading and math scores in the study schools increased
7 to 8 percentile points relative to the scores of students who took
the same tests, leaving the study schools at or near the middle of their
state or national distributions.
The positive impacts on
test scores in the fifth follow-up year reflect an overall increase in
the average reading score from the 37th percentile during the baseline
period to the 44th percentile in the fifth follow-up year. The corresponding
increase in the average math score was from the 45th percentile to the
53nd percentile.
- The impacts were not uniform across students or schools. Students
who would have been in the middle of their school’s test score
distribution without the Accelerated Schools reform were the most likely
to experience test score improvements. The schools that had the lowest
test scores during the baseline period were the most likely to experience
large impacts.
The differential impacts on different groups of students may be attributable
to the fact that the instructional changes made were not substantial enough
to affect the lowest-achieving students. It is also possible that lower-performing
students had higher mobility — that is, were more likely than higher-performing
students not to have attended the Accelerated School during the entire
follow-up period — and therefore received less exposure to the reform.
Nevertheless, the Accelerated Schools initiative improved the performance
of even lower-performing students, because (especially in the initially
lowest-performing schools) even those students who scored in the middle
of their school’s distribution were typically below average nationally
or statewide. Furthermore, the schools that during the baseline period
had the lowest test scores relative to their state or national norms experienced
the largest relative gains.
V.
Implications for the Accelerated Schools Reform
Together these findings demonstrate the potential of the Accelerated
Schools approach, as it was implemented early in its development, to improve
student achievement. Given the difficulty of raising test scores in schools
like those in this study and the relatively low cost of Accelerated Schools,
these findings are noteworthy.
The present study examined the effect of the Accelerated Schools model
on the test scores of all third-grade students who attended the study
schools during the follow-up period. Although the level of student mobility
in these schools was relatively high, this focus was chosen because
it addresses a policy-relevant question: What were the impacts of Accelerated
Schools on third-grade students when the reform was implemented under
real-world conditions — which include, among other things, high
student mobility? Many other studies of education reforms, in contrast,
have asked how the reform under study affects only those students who
remain in the schools long enough to receive a full “dose”
of the reform. Both questions are important, but the present findings
may not be directly comparable to those from the latter type of research;
specifically, the impacts reported here are likely to be smaller than
those found in other studies.
These findings also suggest a need for some operational refinements
of the Accelerated Schools approach, many of which have been put in
place since the schools in this study began implementing the reform.
The lack of emphasis on curriculum and instruction in the initial years
of implementation, the negative impact on achievement in the third follow-up
year, and the appearance of positive impacts only at the end of the
follow-up period all point to a need to focus on curriculum and instruction
earlier in the reform process and to make powerful learning easier to
implement. ASP now provides more technical assistance and encourages
schools to change their curriculum and instruction earlier in the process.
Further, ASP has worked extensively to make the concept of powerful
learning more concrete ― for instance, by providing staff with
specific illustrations of ways in which they could modify their classroom
practices.
Notes:
[1]
Impacts were also estimated using all five years of baseline
data, but this procedure did not materially change the findings and
was not used for reasons described in the following paper: Howard S.
Bloom, “Measuring the Impacts of Whole-School Reforms: Methodological
Lessons from an Evaluation of Accelerated Schools” (New York: MDRC,
2001).
[2] The small changes in the composition of the student body
that were observed, which might also have influenced test scores, were
controlled for through statistical adjustments.
|
