Low-performing public high schools in U.S. cities are often seen as places of little hope. High proportions of students drop out, many students who stay in school are not well prepared for college or the workforce, and efforts to make substantial improvements in ailing schools tend to meet with little success. Two recent trends offer some good news, however. Educators and policymakers — including President Bush, state governors, and foundation and business leaders — have recently recommitted themselves to addressing the challenge of reforming secondary education in urban settings. In addition, a number of comprehensive school reform models have been developed over the past ten years, and some have begun to show evidence of being effective.

This report offers encouraging findings on one such initiative: the Talent Development High School model. Talent Development, which targets some of the most troubled schools in the country, seeks to raise the expectations of teachers and students, with the ultimate goal of preparing all students for postsecondary education and employment. The evaluation took place in the School District of Philadelphia, using a particularly rigorous research design that provides a high level of confidence about Talent Development’s effectiveness — essentially allowing researchers to conclude that changes in student engagement and performance are, indeed, due to Talent Development. In summary, the key findings of this final report from the study are:

• Talent Development produced substantial gains in attendance, academic course credits earned, and promotion rates during students’ first year of high school. These impacts emerged in the first year of implementation and were reproduced as the model was extended to other schools in the district and as subsequent cohorts of students entered the ninth grade.



• Talent Development’s strong positive impacts during the first year of high school are consistent with the model’s intensive initial focus on the ninth grade and its emphasis on combining high-quality curricular and instructional enhancements (including offering transitional math and English courses, creating teaching teams, and providing ongoing coaching for teachers) with pervasive structural reforms (including developing small learning communities and using extended class periods) aimed at building supportive and personalized learning environments.



• The improvements in credits earned and promotion rates for ninth-graders were sustained as students moved through high school. Improvements in student performance on the eleventh-grade state standards assessment began to emerge for later cohorts. There are also early indications that Talent Development is improving graduation rates.

Even with Talent Development’s substantial and persistent positive impacts, the schools still have a long way to go to achieve the initiative’s vision of preparing all students for graduation, postsecondary education, and employment. For instance, even in the Talent Development schools in Philadelphia, more than half of first-time ninth-grade students will not be ready to graduate in four years. Also, because this evaluation focuses on Talent Development’s initial scaling-up effort in the School District of Philadelphia, it is not clear what would be required to produce the same effects in a larger number of schools and in more diverse contexts.

How Was Talent Development Implemented in Philadelphia?

Talent Development was initiated in 1994 through a partnership between the Center for Research on the Education of Students Placed At Risk (CRESPAR) and Patterson High School in Baltimore, Maryland. In 1998, CRESPAR, in collaboration with the Philadelphia Education Fund, a local educational intermediary organization, began Talent Development’s first and most ambitious scaling-up effort in Philadelphia.

Throughout the 1990s, nonselective high schools in Philadelphia faced a growing number of challenges. More than three-quarters of students in the district entered the ninth grade with reading and math skills below grade level, and over half could be considered chronic absentees (that is, students who miss an average of one of every five school days each year). Moreover, each year, fewer than two-thirds of ninth-graders were promoted to the tenth grade. For those who continued on to the upper grades, only about 10 percent performed at or above grade level on standardized state tests, and less than 40 percent were on schedule to graduate four years after starting high school.

As of the 2003-2004 school year, seven of the district’s 22 nonselective high schools were implementing the model. This report focuses on the five schools that began using the model first. Following are the key features of the scaling-up process in these five schools:

• The Ninth Grade Success Academy was the most strongly and consistently implemented element of the Talent Development model.

The schools and CRESPAR made immediate changes to both the structure and the instructional core of the entire ninth grade in each high school. This included relocating all ninth-grade classes to a single floor or wing and creating “learning communities” — small, self-contained academic teams of 100 to 125 students taught exclusively by the same four or five teachers. Each school modified its daily schedule to include blocks of four 80- to 90-minute classes and changed the sequence of courses to allow students to take “double doses” of math and English over the course of the school year (that is, to take CRESPAR’s first-term “transition” courses in math and strategic reading as electives in order to prepare for courses required in the second term, including algebra and English language arts). Each school also provided students with a “Freshman Seminar,” developed by CRESPAR and designed to help students develop solid study skills and to focus on personal and social adjustment issues.

• The schools made more limited progress in transforming the upper grades into Career Academy programs that were fully aligned with the principles and structures of the Talent Development model.

During much of the period covered by this evaluation, CRESPAR and its partners worked to refine the model for students in grades ten through twelve — centered around Career Academies, which combine the structure of small learning communities with curricular choices built around broad career themes. While many of the high schools already had existing Career Academies, many did not have cohesive small learning communities, and the curriculum components were of uneven quality. Specialized courses in math and English, team-teaching, extra supports for struggling students, and teacher professional development were phased in and strengthened for the upper grades during the later stages of the follow-up period for this study.

• Implementation teams at each school, professional development opportunities, and the expertise of CRESPAR’s staff were important facets of the implementation process.

In each school, a small team consisting of a part-time coordinator and part-time curricular coaches helped establish and maintain the model. These teams were supported by a full-time coordinator for the district and by curriculum developers and school reform experts at CRESPAR.

• Implementation of Talent Development was sanctioned by the district, but it received neither formal endorsement nor direct institutional support.

The lack of formal district endorsement, along with variation in the context and in the operational support provided at individual schools, led to variation in the quality and depth of implementation of Talent Development across schools and over time. Nevertheless, each of the five schools that are the focus of this report was able to implement the model with a reasonable level of fidelity and intensity.

How Was the Impact of Talent Development Evaluated?

MDRC used a research method called a “comparative interrupted time series analysis” to estimate the effect of Talent Development on student outcomes. The first step in estimating impacts with this design is to measure the change at Talent Development schools in a given student outcome, after the school began using the model, relative to the average outcome during a pre-implementation baseline period. This estimation represents how student performance changed in the presence of Talent Development. The next step is to measure the corresponding change during the same period for similar schools not implementing the model. This measurement provides an estimate of how student performance would have changed at the Talent Development schools in the absence of the reform. The difference between these two changes is an estimate of the impact of the intervention — what Talent Development caused to happen.

The design for this evaluation is particularly rigorous, addressing many concerns typically raised about research that does not use random assignment. While no quasi-experimental methodology irrefutably establishes causality, this version of the comparative interrupted time series method provides a strong basis on which to attribute changes in student performance to Talent Development.

Did Talent Development Make a Difference?

The report focuses on program impacts for three cohorts of first-time ninth-graders from each of five Talent Development high schools. It includes findings from these students’ first year of high school and follows at least one cohort from each school for a full three years to the point at which they should be in eleventh grade and taking the state’s standardized test in math and reading. The study was also able to follow ninth-graders from two of the schools through four years to examine Talent Development’s impact on graduation rates. The findings presented in the report support the following conclusions:

• For first-time ninth-grade students, Talent Development produced substantial gains in attendance, academic course credits earned, and promotion rates during the students’ first year of high school.

Figure ES-1 provides a summary of the key impact findings. The solid bars represent the changes in key outcomes for the Talent Development schools between the baseline period and the follow-up period. The white bars represent changes in key outcomes for the comparison schools during the same period. The difference between the two bars represents the impact of Talent Development. The numbers below the bars represent the baseline averages for key outcomes (such as attendance rate and basic academic curriculum completed) for first-time ninth-grade students in Talent Development high schools — that is, the performance levels that Talent Development had to build upon.

The first set of bars in the figure shows that attendance rates in the Talent Development schools improved by an average of about 5 percentage points, while the rates remained constant in the comparison schools. Talent Development’s estimated impact on the attendance rates of first-time ninth-graders was 5 percentage points — or an average increase of about nine extra school days per year for each student in a Talent Development high school.

Beyond encouraging students to attend school more regularly, a major goal for Talent Development is to help ninth-graders complete a more rigorous complement of courses during their first year of high school. Figure ES-1 shows that the Talent Development schools consistently outpaced their non-Talent Development counterparts in increasing the percentage of students who completed what could be considered a basic academic curriculum: earning at least five credits during the school year, with three of those credits being in math, English, and science. On average, Talent Development increased the percentage of students completing a basic academic curriculum by about 8 percentage points over what was achieved in the non-Talent Development schools. Although not shown in the figure, Talent Development also increased total credits earned during the ninth grade by about two-thirds of a full-year credit.

Most notably, Talent Development produced a substantial increase in the proportion of students who earned a credit in algebra — nearly 25 percentage points. Algebra is a critical “gate-keeping” course, one that is usually required of students both for high school graduation and for admission to college. For a typical class of about 500 first-time ninth-graders, the estimated impact of nearly 25 percentage points means that Talent Development adds nearly 125 students to the rolls of those who earn a credit in algebra each year.

Finally, Talent Development also improved the rates at which students were promoted from the ninth grade to the tenth (including both end-of-year and mid-year promotions). Figure ES-1 shows that promotion rates in Talent Development schools rose by nearly 10 percentage points after the program began implementation. During the same time period, these rates rose by only about 2 percent in the comparison schools. The resulting impact of 8 percentage points on promotion to the tenth grade means that an additional 40 ninth-graders per school, per year, made the transition from the ninth to the tenth grade — which is generally considered the weakest point in the educational pipeline.

• The impacts on credits earned and on promotion rates were sustained as first-time ninth-graders moved through high school.

Figure ES-1 also shows that Talent Development improved the rate at which students were promoted to the eleventh grade by about 6 percentage points. Though not shown in the figure, Talent Development also sustained its impact on the accumulation of key academic course credits through the eleventh grade. The model produced a 10 percentage point increase in the percentage of students who had earned a minimum of three math and three English credits — a key threshold for staying on course for graduation.

• Talent Development produced marginal improvements in math test scores among early cohorts of eleventh-graders, but stronger improvements occurred for later cohorts of students who were exposed to a more intensive version of the model.

For the first cohorts of eleventh-grade students in the Talent Development high schools, the model produced a modest increase of 6 percentage points in the percentage of students who scored at the basic level or above on the math portion of the state’s standardized assessment. For later cohorts of eleventh-grade students in two of the Talent Development high schools, the model substantially increased the average scaled scores in reading and math — by effect sizes of 0.38 and 0.65, respectively.

• Based on evidence from only the first two schools to implement the model, Talent Development appears to have produced positive impacts on high school graduation rates.

Because they had been working with Talent Development for a full five years, the first two high schools to implement the model provide the opportunity to examine impacts on high school graduation rates. For the first two cohorts of first-time ninth-graders in these high schools, Talent Development improved the likelihood of graduating on time by about 8 percentage points. In other words, for a typical entering ninth-grade class of 500 students, Talent Development was able to produce an average of about 40 new graduates per year.

• Although the likelihood of repeating the ninth grade declined in Talent Development high schools (due to the model’s impact on promotion rates), the results for those students who did need to repeat the ninth grade were mixed.

Prior to Talent Development, about one-third of the ninth-grade students in the study schools were repeating the grade for at least the first time. As noted above, Talent Development did increase the rate at which first-time ninth-graders were promoted to the tenth grade. Among those who still needed to repeat the grade, Talent Development produced an increase of more than 5 percentage points in average attendance rates. Also, the Talent Development high schools saw notable increases in the credits earned by repeating ninth-graders — yet these trends were present in the non-Talent Development schools as well. Overall, however, Talent Development actually increased the likelihood that students who repeated a full year of ninth grade would leave the school system before the end of their fourth year of high school.

Taking Stock of the Results

Talent Development produced substantial and pervasive improvements in outcomes for first-time ninth-grade students in very low-performing high schools. In a high school of 500 first-time ninth-graders, Talent Development adds about nine days of school attendance for each student and helps an extra 125 students pass algebra, an extra 40 students achieve promotion to the tenth grade, and an extra 40 students graduate on time.

Nevertheless, two important cautions are worth noting. First, the schools that have been using the model still have a great deal left to accomplish if they are to reach the model’s aspiration of preparing all students for graduation, postsecondary education, and employment. Even in a Talent Development school, a typical ninth-grader will still miss about 40 days of school, nearly a third will not be promoted to the tenth grade, and more than half will not be ready to graduate within four years. Thus, even successful interventions like Talent Development still need much more power. Toward that end, the Talent Development model continues to evolve by strengthening the upper-grade components, particularly by extending curricular and instructional reforms to tenth and eleventh grades.

Second, achieving these initial positive results required significant extra funds and very demanding changes to school organization, instruction, and teacher support. The estimated additional cost of operating Talent Development is approximately $250 to $350 per student, per year. In Philadelphia, the costs were covered by federal grants, in-kind and direct contributions from the school district, and other funds available to CRESPAR. Despite the financial support for Talent Development from the district, however, these impressive results were accomplished without the district’s formal endorsement and without deeper institutional support. The effectiveness of Talent Development and other comprehensive school reforms is likely to be enhanced and sustained when school districts focus staffing and leadership decisions on specific school improvement strategies and marshal the funding and resources to support them.

In conclusion, the findings in this report provide encouraging evidence that real improvements can be made in some of the lowest-performing high schools in the country — if there exists a sustained investment in developing the skills to deal with poorly prepared students and weak learning environments, and if that investment is built on reasonable fidelity to the tenets and components of a well-conceived reform approach.