About MDRC
Cullinan directs MDRC’s Multiple Measures Assessment Project, which, in collaboration with the Community College Research Center, is evaluating alternative placement systems that use multiple measures in addition to tests. These new placement methods are being developed to determine whether more students can be placed into courses that are appropriate for their levels. He is also the cost analyst for this project as well as for the Dana Center Math Pathways project, which aims to allow community college students who place into developmental mathematics courses to complete in one academic year a credit-bearing, transferable mathematics course relevant to their future major or career goals. He is data manager and impact analyst on the Text Ed demonstration project, which is evaluating the effectiveness of a low-cost enhancement to Educational Opportunity Center services: a systematic set of text messages that include timely, personalized information concerning college enrollment activities and deadlines, as well as resources for overcoming common barriers to enrollment. Before joining MDRC, Cullinan worked at the Center for Research on Children in the U.S. at the Georgetown Public Policy Institute. He has a master’s degree in economics from Virginia Commonwealth University.
-
MDRC Publications
ReportA Study of College Transition Text-Based Messaging
May, 2023Many underserved groups face barriers to college enrollment. This study evaluated a program that supplemented federal supports for these groups through text messages about securing financial aid, completing college enrollment, and navigating other barriers. The study found that adding the messaging program did not increase rates of college enrollment.
BriefLessons from the Field
April, 2023MDRC is partnering with the Texas Higher Education Coordinating Board to provide information about evidence-based programs backed by past research to the Student Success Program Inventory, an inventory of existing student success programs in Texas. This brief summarizes some of the information they will use in that effort.
BriefAn Exploratory Study of Student Outcomes and Placement Practices
June, 2022Informed self-placement (ISP) helps college students determine whether they are ready for entry-level college courses or need remedial education first. This brief explores the potential of ISP to improve students’ access to college-level courses and gives colleges an opportunity to consider placement-method changes that may boost student success.
BriefMay, 2022Multiple measures assessment is a more reliable method than a single placement test to assess whether incoming students have the literacy and numeracy skills required for college-level courses. This brief summarizes the research on multiple measures assessment and offers recommendations for states interested in its implementation.
BriefJanuary, 2022Two experimental studies examined multiple measures assessment (MMA), in which colleges use alternative measures (like high school GPA) rather than just standardized test scores, to assign students to developmental or college-level courses. Students placed using MMA were more likely to complete college-level courses. This brief offers recommendations for other colleges.
ReportThree-Semester Findings from an Experimental Study of Multiple Measures Assessment and Placement
December, 2021Some students are referred into developmental (or remedial) education inappropriately when placed using only standardized placement tests. When multiple measures assessment was used, students in Minnesota and Wisconsin were more likely to enroll and pass college-level math and English courses within three semesters. The additional cost of this alternative assessment averaged $33 per student.
ReportThree Years of the Detroit Promise Path Program for Community College Students
March, 2021This program combines a tuition-free scholarship with additional forms of support, such as a campus coach and personalized communications, to keep students on track to graduate. A three-year evaluation shows that the program helped students stay enrolled in school and earn more credits, but had no impact on degrees earned.
BriefWhat Colleges Need to Know About Multiple Measures Assessments
February, 2021Colleges often use standardized testing to determine whether incoming students need developmental courses. But those tests do not predict students’ college readiness accurately. This brief explains how the use of alternative multiple measures can improve placement decisions, increase college-level course pass rates, and boost progress toward graduation.
ReportImpact Findings From an Evaluation of a Multiple Measures Assessment Strategy
October, 2020A random assignment evaluation at seven State University of New York campuses finds that using multiple measures assessments to determine placement in remedial education led to more students being placed in college-level courses, where they did better than their peers who were placed in remedial classes.
Issue FocusThree Insights from Research
June, 2020The COVID-19 pandemic means community colleges cannot administer in-person tests to determine whether students must take developmental (remedial) courses. But some colleges were already using multiple pieces of information for course placement, rather than a single test score. This brief offers three relevant insights from research on these placement systems.
ReportThree-Year Results from the Accelerated Study in Associate Programs (ASAP) Ohio Demonstration
January, 2020This report presents findings through three years from a replication of the City University of New York Accelerated Study in Associate Programs model at three community colleges in Ohio. The Ohio programs nearly doubled degree receipt through three years and led to an increase in transfers to four-year colleges.
ReportEarly Findings from an Experimental Study of Multiple Measures Assessment and Placement
December, 2019This report examines colleges’ use of multiple measures to determine whether students take college-level or developmental education courses, a more accurate method than standardized placement exams. Using additional placement tests, high school transcripts, and student motivation evaluations places more students into credit-bearing courses, improving academic results and college completion rates.
ReportFinal Report on Aid Like A Paycheck
May, 2019This study, implemented at two community college systems in Texas and one in California, tested whether biweekly disbursements of financial aid rather than lump sum payments could help students budget more efficiently and improve their academic and financial outcomes. Overall, this approach did not have substantial impacts on student outcomes.
ReportInterim Findings from the Detroit Promise Path Evaluation
April, 2019The Detroit Promise allows the city’s high school graduates to attend local colleges tuition-free. To that scholarship the Detroit Promise Path adds campus coaches, monthly financial support, enhanced summer engagement, and messages informed by behavioral science. Interim findings about persistence in school, full-time enrollment, and credit accumulation are all positive.
BriefTwo-Year Findings from the ASAP Ohio Demonstration
December, 2018The highly successful Accelerated Study in Associate Programs (ASAP), developed by the City University of New York, has been shown to nearly double graduation rates after three years. This brief presents results after two years from a replication of ASAP at three community colleges in Ohio.
ReportA Guide to Launching a Multiple Measures Assessment System
July, 2018To address underplacement, in which students who could succeed in college-level courses are directed into developmental education, community colleges have begun supplementing the typical placement test with measures like high school GPA and noncognitive assessments. This guide walks colleges through the process and pitfalls of undertaking this kind of reform.
ReportInterim Findings on Aid Like A Paycheck
June, 2017This study examines whether an alternative approach to distributing financial aid — in biweekly payments instead of one or two lump sums — can improve outcomes for low-income community college students. After one semester, the policy reduced students’ debt and use of federal loans but showed little consistent evidence of academic effects.
ReportUsing Behavioral Insights to Increase Incarcerated Parents’ Requests for Child Support Modifications
October, 2016A behavioral intervention provided incarcerated noncustodial parents in Washington with materials about their eligibility for a child support order modification and how to request one. It increased the number of parents requesting a modification by 32 percentage points and the number of parents receiving a modification by 16 percentage points.
ReportApplying Behavioral Insights to Increase Collections
February, 2016Findings from tests in Cuyahoga County, Ohio, demonstrate that low-cost, low-effort behavioral interventions can improve child support payment outcomes. These tests are part of the Behavioral Interventions to Advance Self-Sufficiency project, sponsored by the federal Administration for Children and Families.
ReportUsing Behavioral Economics to Increase On-Time Child Care Subsidy Renewals
November, 2015This study assessed three different behavioral strategies for providers and clients aimed at increasing the timely renewal of child care subsidies, in order to ensure consistent client services. The findings suggest that strategies designed for staff who work directly with clients may be a fruitful area for future work.
ReportUsing Behavioral Economics to Increase Child Support Payments
July, 2015A low-cost behavioral intervention produced a modest increase in the number of parents in Franklin County, Ohio, who made at least one child support payment over four months. This test is part of the Behavioral Interventions to Advance Self-Sufficiency project, sponsored by the federal Administration for Children and Families.
ReportUsing Behavioral Economics to Help Incarcerated Parents Apply for Child Support Order Modifications
September, 2014A low-cost behavioral intervention increased by 11 percentage points the proportion of incarcerated noncustodial parents in Texas who applied for modifications to reduce the amount of their child support orders. This test is part of the Behavioral Interventions to Advance Self-Sufficiency project, sponsored by the federal Administration for Children and Families.
ReportFinal Report
September, 2014This report discusses a pilot project to prepare adult education students in New York City for the new more rigorous GED exam. Revised writing and math curricula were offered to thousands of students, but attendance was erratic. Shorter lesson sequences and support outside the classroom might allow more students to benefit.
ReportLessons from the First Round of Achieving the Dream Community Colleges
April, 2014Launched in 2004, Achieving the Dream is designed to help community colleges collect and analyze student performance data and apply the results to help students succeed. This report offers lessons from the first 26 colleges to join the national initiative, which now includes more than 200 institutions.
Working PaperSeven Years Later
March, 2014This paper presents the long-term effects of a learning communities program. The program’s positive effect on credit accumulation was maintained for seven years, and there is some evidence that graduation rates increased. Economic outcomes are examined, and sobering reflections on detecting effects on economic outcomes in higher education interventions are presented.
ReportSix-Year Effects of a Freshman Learning Community Program at Kingsborough Community College
July, 2012Students who participated in a one-semester learning community, in which small groups of student took three linked classes together and received other extra services, were more likely to have graduated six years later. The program also proved to be cost-effective.
ReportAn Impact Study of a Student Success Course at Guilford Technical Community College
April, 2012A random assignment study of a student success course for developmental students finds positive effects on students’ self-management, self-awareness, and engagement in college. The program had few overall effects on students’ academic achievement, although there were some positive impacts for the first group of students to enter the study.
ReportImpact Studies at Merced College and The Community College of Baltimore County
February, 2012Two colleges implemented semester-long learning communities linking developmental English with a range of other courses. At Merced, learning communities students earned more developmental English credits and passed more English courses than a control group. At CCBC, there were no meaningful impacts on students’ credit attempts or progress. Neither college’s program had an impact on persistence or on cumulative credits earned.
ReportFive Years of Achieving the Dream in Community Colleges
February, 2011This interim report examines the experiences of the first 26 colleges to join the ambitious Achieving the Dream initiative. Launched by Lumina Foundation for Education in 2004, Achieving the Dream helps community colleges collect and analyze student performance data in order to build a “culture of evidence,” enabling the colleges to use that knowledge to develop programs to increase students’ academic success.
-
Other Publications
-
Projects
College administrators are better positioned than ever before to make decisions about adopting programs based on the effectiveness and cost of interventions. There is, however, a third piece of information critical to decision-making: the amount of revenue an intervention would generate at their college because of increased student retention...
MDRC’s Applying the Evidence project is a partnership with the Minnesota State Colleges and Universities system (Minnesota State) to expand the implementation of evidence-based student success strategies across the Minnesota State system, which currently includes 30 community colleges and seven universities.
In partnership with system-...
Dan Cullinan, Rashida Welbeck, Alyssa Ratledge, Tiffany Morton, Erika B. Lewy, Dorota Biedzio Rizik, Stanley DaiCollege students who place into developmental (remedial) education are substantially less likely to graduate than students who place into college-level courses. Most students are directed into developmental courses based on placement test scores. But large-scale studies have indicated that these test scores misplace substantial numbers of students — in other words, for...
Alexander Mayer, David Navarro, Caitlin Anzelone, Evan Weissman, Dan Cullinan, Elena Serna-Wallender, Stanley Dai, Sumner PereraThe college enrollment process is complex and includes many steps. Recent research has shown that short, action-oriented text messages can help people focus on critical tasks at the right times. Building on this promising research, the Text Ed project will develop and test a strategic text messaging intervention at Educational Opportunity Centers (...
Alexander Mayer, Dan Cullinan, Evan Weissman, Michael J. Weiss, John Diamond, Rashida Welbeck, Elizabeth Zachry RutschowThe Center for the Analysis of Postsecondary Readiness (CAPR) conducts research to document current practices in developmental English and math education across the United States and to rigorously evaluate innovative assessment and instructional practices. CAPR, led by MDRC and the Community College Research Center, is funded by the federal Institute of Education Sciences.
Dan Cullinan, Amanda Grossman, Elizabeth Zachry RutschowToo many of the nation’s young adults struggle to obtain the educational credentials needed to be successful, productive members of society. When this study was under way, 39 million adults in the United States lacked a high school diploma or its equivalent; moreover, every year, more than 1.3 million students dropped out of high school. Roughly two-thirds of them...
Can existing financial aid programs do more to help low-income college students achieve academic success? MDRC is conducting a large-scale evaluation of Aid Like A Paycheck, a new program based on a simple yet potentially transformative idea: After the college receives payment for tuition and fees, disburse remaining financial aid to students...
Many social programs are designed in such a way that individuals must make active decisions and go through a series of steps in order to benefit from them. They must decide which programs to apply to or participate in, complete forms, attend meetings, show proof of eligibility, and arrange travel and child care. Program designers often assume that individuals will...
Dan Cullinan, Alexander Mayer, Michelle Ware, Michael J. Weiss, Evan Weissman, Alyssa Ratledge, Jedediah J. TeresA postsecondary credential has become increasingly important in the labor market, and college attendance has grown. Unfortunately, college completion remains less common, particularly in community colleges, which serve many low-income and academically underprepared students who often need remedial (developmental) courses. Finding ways to increase the rates of...
Michael J. Weiss, Susan Scrivener, Colleen Sommo, Dan Cullinan, John Diamond, Alyssa Ratledge, Jedediah J. TeresCommunity colleges, which tend to be accessible and affordable, serve as a critical resource for low-income individuals striving to improve their prospects in the labor market and life. However, a variety of factors, ranging from a lack of financial aid to inadequate student services and poor developmental classes, can impede students’ progress. Many students stop...
Community colleges enroll almost half of all U.S. undergraduate students, yet the majority of these students leave without earning a degree or certificate or transferring to another institution to continue their studies. As a result, they risk losing the opportunity to learn and to earn a livable wage.
Achieving the Dream: Community...