High school reform has moved to the top of the education policy agenda, commanding the attention of the federal government, governors, urban school superintendents, philanthropists, and the general public. All are alarmed by stubbornly high dropout rates, by the low academic achievement of many high school students, and by the large numbers of high school graduates who are required to take remedial classes in college. These trends disproportionately affect urban and certain rural areas and minority groups: The most troubled high schools are concentrated in about 50 large cities and 15 primarily southern and southwestern states, and the majority of their students tend to be African-American or Hispanic.

This is the first in a series of reports summarizing and synthesizing what has been learned from rigorous and large-scale evaluations of high school reform initiatives. It discusses three comprehensive initiatives — Career Academies, First Things First, and Talent Development — that have grappled with the challenges of improving low-performing urban and rural schools. Together, these three interventions are being implemented in more than 2,500 high schools across the country, and various components of these models are being used in thousands more schools. This report focuses almost exclusively on MDRC research, but subsequent reports will synthesize lessons from additional studies of high school reform conducted by MDRC and others. While aiming to be useful to researchers, this series of reports is directed primarily toward policymakers, practitioners, and others who must make hard choices about how to change high schools.

Each of the three programs that MDRC studied involved multiple components. (Table ES.1 briefly describes the programs and their evaluations.) Each program, too, featured a philosophy or theory of action that linked the various components into a coherent whole that program developers believed would be more than the sum of its parts, and the developers offered considerable technical assistance about how best to put the components in place. MDRC’s evaluations of these programs built on rigorous research designs using comparison or control groups, and they provide unusually strong evidence about the interventions’ effects on attendance, academic achievement, persistence in school through graduation, and postsecondary education and labor market outcomes. Importantly, these impacts reflect the combined effects of all the components, packaged in a particular way by the programs’ developers. For districts and schools interested in replicating a comprehensive school reform, turning to one of MDRC’s reports on these models would be a good first step.

Some policymakers and practitioners, however, may not want to join forces with a multicomponent comprehensive school reform model like the ones that MDRC studied; instead, they seek informed advice about how to fill in the missing pieces in their current reform strategies. For them, this report takes a different path. It discusses five major challenges associated with low-performing high schools and offers lessons addressing each. Looking inside the “black box” of the three comprehensive reforms, the report seeks to draw reasoned conclusions about which particular aspects of the reforms made them effective (or, in some cases, proved ineffective). It tries to link particular outcomes to particular inputs, using available evidence from MDRC’s evaluations, including analysis of student records, teacher and student surveys, and field research, along with the program developers’ own theories of change, where possible. At the same time, it acknowledges that each program may be more than the sum of its components and that conclusions about particular components of the initiatives can never be as solidly grounded as conclusions about the effects of the programs as a whole. Thus, while phrases like “appears to” and “suggests that” are not completely satisfying, they remind readers that the lessons go beyond the bounds of what is known with confidence.

Because of these methodological issues, lessons in this report should be viewed as judgments, not facts. Almost all the judgments are grounded in evidence, although that evidence is thick in some cases, thinner in others. In a few instances, these lessons represent the assessments of the program developers or of researchers who have studied the programs for many years. In this Executive Summary, an effort has been made to give the reader a sense of the evidence on which each lesson is based; for further details, readers are referred to the body of the report.

Challenge 1

Creating a Personalized and Orderly Learning Environment

A positive school climate — where students and adults know each other well and where adults express care and concern for students’ well-being, intellectual growth, and educational success — is a key motivational element in the learning process for adolescents. But the large size of many low-performing high schools leaves many students, especially those who are less academically successful, feeling lost and anonymous and prevents the development of an atmosphere conducive to learning. This problem may be exacerbated for ninth-graders leaving behind the more family-like environment of middle school — a critical issue because students attending low-performing schools who do not complete ninth grade successfully and on schedule are at greatly heightened risk of dropping out altogether. The MDRC studies of the three reform models suggest that changes in the structure and functioning of large high schools can help remedy the impersonality of large high schools.

• Student survey data suggest that small learning communities — groups of students who share the same cadres of core-subject teachers — make students feel known and cared about by their teachers. Students in First Things First schools registered higher levels of perceived support from their teachers after the demonstration was implemented than they had before it was put in place, and Career Academy students reported higher levels of teacher support than members of a control group.



• The experiences of First Things First in Kansas City, Kansas, and of Talent Development in Philadelphia indicate that both small learning communities that encompass all four grade levels and separate Freshman Academies followed by communities for upperclassmen can play a role in increasing attendance and reducing dropout rates. While feeling connected to teachers and classmates is only one factor that promotes attendance and persistence, both interventions, with their different small learning community structures, had positive effects on these outcomes.



• The separate Freshman Academy structure may have played a key role in helping more ninth-graders succeed in the critical first year of high school. Students in Talent Development’s Ninth Grade Success Academies received special attention from their teachers, and their rates of attendance and on-time promotion were higher than those of ninth-graders in comparison schools.



• Faculty advisory systems can give students a sense that there is an adult in the school looking out for their well-being. Almost three-quarters of First Things First students reported on surveys that their advisor was either “very important” or “sort of important” in giving them someone to talk to when needed, helping them do better on schoolwork, and recognizing their accomplishments. Training helped family advocates perform their roles more effectively.



• Implementing small learning communities is not easy. School administrators and program operators report that scheduling classes to ensure that they contain only teachers and students within the same small learning community can present a major challenge. This problem is especially marked for students in the upper grades, who may want to take electives offered only by communities other than the one to which they belong.



• Implementing small learning communities is likely to improve the climate of schools but will not, in and of itself, increase student achievement. It may help to do so, but the studies do not provide conclusive evidence on this point. All three initiatives that were studied involved small learning communities. Talent Development improved eleventh-grade math and reading test scores for students where the intervention had been in place longest (although other elements of the model undoubtedly also contributed to these results). By contrast, Career Academies had no effect on achievement, and First Things First was effective in boosting achievement only in the first district where it was implemented and in one school in a second district.

Challenge 2

Assisting Students Who Enter High School with Poor Academic Skills

Large numbers of students enter urban high schools poorly prepared for academic success. The Ninth Grade Success Academy — the centerpiece of the Talent Development model — tackled the problem of low achievement among entering ninth-graders head-on through interconnected changes in scheduling and curricula and produced positive results for many students. The Talent Development experience suggests the following lessons:

• A double-blocked class schedule is useful because it permits students to attempt and earn more credits per year than other scheduling arrangements. In contrast to a traditional schedule (entailing daily 50-minute classes) or a single-blocked schedule (involving 80- or 90-minute classes meeting every other day), a double-blocked schedule calls for classes that meet daily for extended periods. Because double-blocked classes can cover in a single semester what would normally be a year’s worth of material, students in Talent Development schools could earn four full course credits each term and eight credits each year, compared with the six or sometimes seven credits per year that students would receive in schools following a traditional schedule.



• Semester-long, intensive “catch-up” courses that shore up ninth-grade students’ skills in reading and mathematics appear to help students succeed in the regular curriculum, with gains in credits earned being sustained over time. The catch-up courses in Talent Development awarded elective credits and were designed to precede and prepare students for college preparatory classes in English and algebra. (The double-blocked schedule allowed the catch-up and regular classes to be sequenced in this way.) First-time ninth-graders in the Talent Development schools were significantly more likely than their counterparts elsewhere to earn one or more credits in English and algebra. For these students, too, the intervention increased the total number of credits earned in the first three years of high school.



• The structured curriculum of catch-up courses, combined with longer class periods, may have helped ensure that students spent more time “on task” in these classes. More time in the classroom may not in itself be enough to improve student achievement; what appears to matter is that the extra time be used to maximize learning. Most First Things First schools made substantial progress in implementing longer English and math class periods. However, no special curricula were in place during the period under study (a situation that First Things First has subsequently addressed), and most expansion-site schools did not register increases in student achievement.



• Little is known about how best to assist and prevent dropping out among those students who struggle the most in ninth grade. While Talent Development increased the rate of promotion to tenth grade, those students in Talent Development schools who were required to repeat a full year of ninth grade were more likely to drop out of high school than their counterparts in other schools. Different grouping arrangements and modes of instruction may be needed for such students.

Challenge 3

Improving Instructional Content and Practice

Teachers in schools serving disadvantaged populations are often less experienced and less knowledgeable about the subjects they teach than teachers in more affluent communities. The high school reforms that MDRC studied have addressed questions about how to improve the content and delivery of what is taught through the use of new curricula and through professional development. While only limited data are available linking instructional improvement efforts to changes in student outcomes, the experiences of the program developers and of the participating schools and teachers suggest a number of operational lessons about putting instructional improvement efforts in place.

• It may not be realistic to expect teachers to create their own curricula reflecting the themes of their small learning communities; instead, they are likely to benefit from well-designed curricula and lesson plans that have already been developed. First Things First’s developers expected teachers to integrate the theme of their small learning communities into their classes, but teachers said that they had neither time nor training to do this, and field research observations and interviews indicate that thematic instruction was uncommon. Similarly, teachers of academic subjects in the Career Academies generally followed the standard curriculum, rather than creating lessons that reflected their Academy’s occupational focus.



• Good advance training and ongoing coaching can help teachers make better use of even well-designed curricula. Teachers in Talent Development schools who received training on teaching the catch-up courses reported that the training had helped them deliver their lessons more effectively.



• There is suggestive evidence that student achievement may be enhanced by professional development activities that involve teachers working together to align curricula with standards, review assignments for rigor, and discuss ways of making classroom activities more engaging. The expansion-site high school that the First Things First developer and researchers agreed had made the most progress in developing “professional learning communities” of teachers — who met regularly to discuss pedagogy — showed positive impacts on reading achievement.



• Both academic departments and small learning communities should be regarded as key venues for instructional improvement. First Things First developers initially sought to focus instructional improvement efforts on the small learning community. But they came to realize that while the small learning community is an appropriate setting for professional development directed toward improving pedagogical methods, teachers look to other department members as repositories of content expertise and, therefore, that departments should be incorporated into initiatives to improve instructional quality.



• If administrators want teachers’ meetings to focus on instructional improvement, they must both provide guidance about how to do this and follow up to ensure that meeting time is used productively. Researchers’ observations of teachers’ meetings in small learning communities revealed that, without specific direction about how to spend their time together, teachers talked mostly about matters unrelated to instruction (such as discipline issues, individual students’ personal or academic problems, or planned small learning community field trips or parties). When administrators issued guidelines specifying that meetings were to focus on instruction — and when they sat in on these meetings — discussion centered instead on pedagogical concerns.

Challenge 4

Preparing Students for the World Beyond High School

Students in low-performing schools need special assistance in preparing for postsecondary education and for better-paying jobs. Among the initiatives considered in this report, Career Academies are most clearly oriented toward the goal of helping students prepare for a productive future after they leave high school. Both Talent Development and First Things First seek to improve academic achievement and graduation rates — which presumably would increase opportunities for students in the labor market and in postsecondary education — but neither has a strong postsecondary thrust, nor did the MDRC studies follow students beyond high school. The Career Academies study, which has the advantage of long-term follow-up, suggests the following lessons:

• Earnings impacts for young men in Career Academies appear to be linked to career awareness activities and work internships during high school. Young men in the Career Academies group earned over $10,000 more than members of a control group over the four-year period following their high school graduation. Participation in career awareness sessions and work internships most clearly distinguished the high school experiences of Career Academy students from those of their counterparts who were not in the Academies.



• The potential benefits of partnerships between high schools and employers can be more fully realized when these partnerships are more structured and when schools can designate a full-time, nonteaching staff person to serve as a liaison with employers. Students in Career Academies with more structured partnerships and with full-time liaisons reported higher levels of participation in career awareness and work-based learning activities than did students in Academies where arrangements were less formal and where liaisons also had teaching responsibilities.



• It may be necessary to improve the academic component of Career Academies in order to raise students’ achievement on standardized tests and help them secure admission to college. Students in the Career Academies did not have higher academic achievement or graduate from high school at higher rates than their non-Academy counterparts, nor were they more likely to enroll in college or earn a credential. Field researchers found that core-subject courses in the Academies were very similar to courses in the rest of the school.

Challenge 5

Stimulating Change

Introducing change into high schools and making it stick goes beyond the discrete challenges discussed above. The following implementation lessons primarily reflect the perceptions and judgments of program developers and researchers. The lessons are likely to apply not only to ambitious and large-scale reforms like the ones studied here but also to less far-reaching efforts to introduce change into overstressed high schools.

• Creating effective change demands an investment of personnel resources. Whether personnel come from inside or outside a school or district, they must be skilled in designing reforms, putting them in place, and monitoring ongoing operations.



• In deciding whether to adopt a comprehensive reform model or add new components to existing programs, school and district administrators should consider the adequacy of what is already in place and the capacity of local personnel to envision and implement change. The fewer the reform elements already in place and the more limited the capacity of local staff, the more sense it may make for administrators to turn for assistance to the developers of comprehensive models.



• Strong support of the initiative by the school district helps to ensure effective implementation and the reform’s continuing existence. The contrasting experiences of First Things First in Kansas City, Kansas, and of Talent Development in Philadelphia exemplify this point. In Kansas City, the central office leadership both exerted pressure on the schools to operate in conformity with First Things First guidelines and supported the schools’ efforts to do so; close and consistent monitoring was a hallmark of the district’s efforts. While the School District of Philadelphia initially welcomed Talent Development, it never formally endorsed the initiative or gave it support, and some of its actions (for example, the introduction of a new standardized curriculum and reductions in funding) undercut the program model.



• It is important for policymakers and administrators to avoid jumping from one reform to the next; instead, they should stay the course until initiatives have been put in place long enough and well enough for their effectiveness to receive a fair test. Research has shown that comprehensive reforms in place for five years or more had stronger impacts than those with briefer periods of implementation. Extended research follow-up may also be important: In the Career Academies evaluation, for instance, the initiative’s substantial effects on postsecondary employment were evident four years after students’ scheduled graduation from high school.



• It is important to have high ambitions but also reasonable expectations about the size of impacts that reforms can produce. Careful evaluations of reform efforts seldom find large and dramatic effects. But even impacts that appear to be small can nonetheless be important. For example, Talent Development’s 8 percentage point effect on the rate of promotion from ninth to tenth grade means that hundreds of freshmen in Talent Development schools did not have to repeat the year and were at much lower risk of dropping out of school altogether.

Conclusion

The larger lesson of this report may be that structural changes to improve personalization and instructional improvement are the twin pillars of high school reform. Small learning communities and faculty advisory systems can increase students’ feelings of connectedness to their teachers. Especially in interaction with one another, extended class periods, special catch-up courses, high-quality curricula, training on these curricula, and efforts to create professional learning communities can improve student achievement. Furthermore, school-employer partnerships that involve career awareness activities and work internships can help students attain higher earnings after high school.

A further message is that students who enter ninth grade facing substantial academic deficits can make good progress if initiatives single them out for special support. These supports include caring teachers and special courses designed to help entering ninth-graders acquire the content knowledge and learning skills that they missed out on in earlier grades.

Whether districts and schools adopt a comprehensive reform initiative like the ones MDRC studied or put together the elements of a comprehensive intervention on their own, much has been learned about what is needed — and what seems to work. What remains is to make sure that practitioners have the support they need to put that learning into practice.