An Experienced School Support Organization at Scale: A Study of The Urban Assembly Network

Project

Overview

Project Overview

Amid modest increases in high school graduation rates for all students, disparities still exist for important student groups, such as students of color and students from low-income families. In response to this equity issue, there has been a growth in third-party “school support” organizations like Urban Assembly. Since 1997, Urban Assembly has supported New York City schools with the mission to improve all students’ economic and social mobility, with a focus on “meeting students where they are,” even when they enter high school performing below grade level.

Under contract with the New York City Department of Education, in the 2019–2020 school year, Urban Assembly supported 23 “unscreened” schools serving roughly 9,000 middle and high school students throughout Manhattan, Brooklyn, and the Bronx. Urban Assembly provides their schools with professional development to school-based staff using two formats: professional learning communities (PLCs) and coaching. They also offer schools five main program areas of support: academic, leadership, social emotional learning, postsecondary readiness, and alumni success.

Despite the significant monetary investment in school support organizations, such as Urban Assembly, there is limited peer-reviewed research on their effectiveness. This study aims to fill that gap and yield lessons for other school support organizations on best practices. Additionally, this study will evaluate the effects of the Urban Assembly school support network on the academic and socio-emotional outcomes of students, as well as apply the most rigorous causal approach to understanding what works to improve their outcomes.

STATUS: Current

Basics

Issue: P-12 Education

Focus Area: High School, Institutional Reform, Transition to Postsecondary

Population: Children and Schools

Focus Population: High School Students, Students Experiencing Academic Struggles, Teachers/School Administrators

Locations: New York

Agenda, Scope, <br> & Goals

Agenda, Scope, and Goals

The research team is conducting a comprehensive study of the landscape of school support organizations working with New York City Department of Education public schools, the implementation of the Urban Assembly model, and the effects of the Urban Assembly model on students. More specifically:

The landscaping strand of work will explore the following research questions:

What is the landscape of the larger system of high school support in the New York City Department of Education?
Does the implementation of the Urban Assembly model create differences from other school and student support practices in the New York City Department of Education?

The implementation strand of work will explore the following questions:

To what extent are the core program areas of Urban Assembly’s model being offered as intended? To what extent do schools participate in the core program areas offered by Urban Assembly as intended?
What are the conditions that facilitate or challenge Urban Assembly’s ability to support their schools?
Within Urban Assembly schools, how does the organizational structure of the school and its programming support students?
To what extent are the core program areas of Urban Assembly’s model (academic, leadership, social emotional learning, postsecondary readiness, and alumni support) being offered as intended in schools? To what extent do students experience the core program areas of Urban Assembly’s model?
What are the conditions that facilitate or challenge Urban Assembly schools’ ability to support their students?

The impact strand of work will explore the following questions:

Treatment Contrast: What is the impact of assignment to an Urban Assembly school on students’ experience of the key features of the theory of change overall and by its core components?
Short-term Impacts: What is the impact of assignment to an Urban Assembly school on students’ outcomes during high school, including: measures of student social and behavioral competencies (including indicators of growth mindset, resilience, response to challenge, and intrinsic motivation) and progress towards high school graduation (including students being on-track at the end of ninth grade, academic credit accumulation and GPA, CTE-specific credit accumulation, New York State Regents examinations, attendance, and mobility)?
Longer-term Impacts: What is the impact of Urban Assembly on longer-term outcomes, including: receipt of New York State diploma; enrollment, persistence, and completion of two-, four-, or six-year college degrees, or license/certificate programs; and postsecondary employment and earnings?
Impact variation: Do Urban Assembly schools with specific features produce different effects?
Cost analysis and effectiveness: How do the costs of Urban Assembly compare to business as usual costs?

STATUS: Current

Basics

Issue: P-12 Education

Focus Area: High School, Institutional Reform, Transition to Postsecondary

Population: Children and Schools

Focus Population: High School Students, Students Experiencing Academic Struggles, Teachers/School Administrators

Locations: New York

Design, Sites, & <br> Data Sources

Design, Sites, and Data Sources

The research team will take advantage of the rich array of data available.

Landscaping. The landscaping strand of work will explore the context for intervention, by capturing the New York City Department of Education’s system of school support organizations. In turn, these data will be used to document the comparable practices available to non-Urban Assembly schools and students.

Implementation. The implementation strand of work will document the implementation of the Urban Assembly model, assess the fidelity of model implementation, and compare the practices of Urban Assembly and its schools to those of other school support organizations and their schools.

Impact. The impact strand of work will be anchored in a high-powered, naturally occurring randomized controlled trial design executed to meet federal What Works Clearinghouse standards, complemented by a uniquely rigorous propensity score matching approach. This design serves to answer questions regarding treatment contrast, short-term impacts, long-term impacts, impact variation, and the cost-effectiveness for policymakers and practitioners who are supporting similar populations.

STATUS: Current

Basics

Issue: P-12 Education

Focus Area: High School, Institutional Reform, Transition to Postsecondary

Population: Children and Schools

Focus Population: High School Students, Students Experiencing Academic Struggles, Teachers/School Administrators

Locations: New York

Funders & <br />Partners

Funders

Institute of Education Sciences, U.S. Department of Education

Partners

Research Alliance for New York City Schools

The Urban Assembly

MDRC Staff

Rebecca Unterman
Senior Associate, K-12 Education
Unterman joined MDRC in 2006 and is a research associate in the K-12 Education Policy Area.
More »
William Corrin
Director, K-12 Education
Corrin came to MDRC in 2004 and has been leading experimental and quasi-experimental evaluations of reforms and interventions at the secondary school level. He is currently leading a national evaluation of Academic Language Interventions.
More »
Emma Alterman
Research Associate, K-12 Education
Alterman is an implementation, qualitative methods, and operations specialist whose expertise includes stakeholder interviews, qualitative analysis, and large-scale, multisite implementation data collection. Her work largely focuses on high school structures and supports and the transition to college and career.
More »
Miki Shih
Research Associate, K-12 Education and Young Adults and Postsecondary Education
Bairstow joined MDRC as a technical analyst in 2018.
More »
Bryce Marshall
Research Assistant, K-12 Education
Marshall is a Research Assistant in MDRC’s K-12 Education policy area, where he works in a variety of roles.
More »