Varying Levels of SUCCESS

group of college students in a classroom

Community colleges and broad-access universities (those with minimally selective admis­sions policies) provide an opportunity for students across the United States to attain postsec­ondary degrees and economic mobility. However, graduation rates from such colleges are of­ten low and there are many obstacles that can be difficult to overcome, especially for students who must balance work or family responsibilities, older students, students from low-income backgrounds, and students of color who face additional systemic barriers. Furthermore, the COVID-19 pandemic introduced unprecedented challenges for college students, making the pursuit of higher education even more difficult.

Since about 2000, researchers have been collecting evidence on what forms of support are effective in helping students earn their degrees. Evidence shows that interventions that in­clude multiple program components that support students over several years are associated with larger impacts on student outcomes. Building on the existing body of research, MDRC designed and is evaluating the Scaling Up College Completion Efforts for Student Success (SUCCESS) program, a multifaceted student support program designed to effectively pro­mote student success and be financially sustainable. SUCCESS combines evidence-based components, including coaches engaged in active outreach to students, monthly financial incentives for students who meet program requirements, strategies to encourage students to enroll full time, and a data-driven program management system.

Starting in 2019, 13 colleges across five states (California, Indiana, Minnesota, New Jersey, and Ohio), along with their state higher education agencies, have worked with MDRC to cus­tomize and launch SUCCESS. (Eleven of the 13 colleges are participating in the random­ized controlled trial.) A previous brief presenting early findings from the first study cohort illustrated that the SUCCESS program in the 2020–2021 academic year, as adapted for the context of the pandemic, had no discernible effect on students’ academic progress. This re­port provides updated insight into the SUCCESS program after one year of participation for the first three evaluation student cohorts, covering fall 2020 through summer 2022. The main implementation finding from that time period is that the program implementation varied by college and term, and did not fully align with the SUCCESS model, largely due to the adapta­tions implemented during the COVID-19 pandemic. However, students who were offered SUCCESS had a different college experience from students in the control group—they were more likely to be told about the importance of full-time enrollment and, on average, they had substantially more contact with their advisors or coaches.

Despite changes in the college experience across the study sites, analyses of academic data show that, on average, there are no discernible positive impacts on persistence or credit accumulation through one year for the full sample. There is, however, evidence that impacts on credit accumulation vary across colleges and cohorts. Exploratory analyses suggest that the quantity and quality of coaching, hearing that full-time enrollment is important, and tak­ing courses in person may all be associated with improved academic outcomes. Given the pandemic’s effect on program implementation and the broader context of students’ lives, it is hard to know whether SUCCESS would have produced stronger effects if implemented as designed outside of the pandemic. Upcoming briefs will include findings from 11 colleges, will include longer follow-up for the initial colleges, and will continue to explore variation in implementation and effects on academics across colleges and entering cohorts.

Sommo, Colleen, Austin Slaughter, Cyrette Saunier, Susan Scrivener, and Kayla Warner. 2023. Varying Levels of SUCCESS. New York: MDRC.