Over the past few decades, a postsecondary credential has become increasingly important in the labor market, and college attendance has become more common. Unfortunately, however, many students leave college before receiving a degree, particularly those who are academically underprepared for college-level work. Many postsecondary institutions operate learning communities to promote students’ involvement and persistence in college. Learning communities typically place groups of students in two or more linked courses with mutually reinforcing themes and assignments. They seek to build peer relationships, intensify connections to faculty, and deepen understanding of coursework. While learning communities are increasingly popular, little rigorous evidence on their effects exists.
As part of the Opening Doors demonstration and evaluation project jointly undertaken by MDRC and the MacArthur Foundation-funded Network on Transitions to Adulthood, six participating colleges operated innovative programs aimed at increasing students’ achievement and persistence and, eventually, their graduation rates and earnings. Kingsborough Community College in Brooklyn, New York — a large, urban college in the City University of New York (CUNY) system — tested a program called Opening Doors Learning Communities. The program placed freshmen, most of whom failed one or more of the skills assessment tests that all incoming students take, into groups of up to 25 who took three classes together during their first semester. It also provided enhanced counseling and tutoring as well as a voucher for textbooks.
This report discusses the program’s implementation and its effects on students up to two years after they entered the study. Using a rigorous research design, MDRC randomly assigned students either to a program group that was eligible for the learning community or to a control group that received standard college courses and services. Any subsequent substantial differences in outcomes can be attributed to the Opening Doors program.
In summary, the key findings from this report are:
- The program improved students’ college experience. Students in the program group felt more integrated and more engaged than students in the control group.
- The program improved some educational outcomes while students were in the learning community, but the effects diminished later. Program group students passed more courses and earned more credits during their first semester.
- The program moved students more quickly through developmental (remedial) English requirements. Students in the program group were more likely to take and pass the college’s English skills assessment tests that are required for graduation or transfer.
- The evidence is mixed about whether the program increased persistence in college. Initially the program did not change the rate at which students reenrolled. At the end of the report’s follow-up period, however, slightly more program group members than control group members attended college.
How Was the Program Implemented?
Opening Doors Learning Communities — operated between 2003 and 2005 — placed groups of freshmen into three linked classes: an English course, usually at the developmental level; an academic course, such as health or psychology; and a one-credit orientation course. The instructor of the orientation course provided enhanced counseling to students, and the program provided enhanced tutoring as well as a voucher for textbooks.
The program was targeted to freshmen who planned to attend college full time during the day and who did not test into English as a Second Language. ESL students were excluded because they participated in another learning community program. For the same reason, students in four “career majors” were excluded for the first year of the study. Over three-fourths of the students were under age 21 when they entered the study. Reflecting the diversity of the student body at Kingsborough, 38 percent identified themselves as black, 27 percent as white, and 20 percent as Hispanic. Almost three-fourths of the students in the study reported that they or at least one of their parents were born outside the United States.
The following key findings on the implementation of Kingsborough’s learning communities program are based on interviews with and surveys of the college’s administrators, faculty, staff, and students.
- The program’s key features were all in place when operations began, and they remained in place throughout the study.
Despite a compressed planning period, Kingsborough’s program was well implemented from the start. This achievement reflects the college administration’s strong commitment to the program and the study.
- The learning communities varied in class size and the degree to which faculty integrated their courses.
Over four semesters, Kingsborough ran 40 learning communities for the study: 31 with developmental English and 9 with college-level English. Owing to several challenges — including the difficulty of predicting how many students would test into each English level — class size varied from 6 to 25 students, with an average of 17.
All instructors developed a new syllabus or revised their regular syllabus for the learning community, and all learning communities had some joint assignments across classes. The degree of integration across the courses varied, however, as did the frequency of joint assignments. The instructors in most learning communities met regularly to discuss student progress and coordinate assignments, but, in a minority of communities, the instructors rarely met. Thus, the study provides a strong test of the structural features of the learning community, but it may not fully detect the effects of tightly integrating course curricula.
How Was the Impact of the Program Evaluated?
As noted above, to determine the effect, or “impact,” of Kingsborough’s program, MDRC assigned students, at random, to either a program group or a control group. Random assignment occurred just before students registered for classes. The study is tracking both groups over time to determine whether the learning community program results in better outcomes for students. Random assignment ensures that the motivation levels and personal characteristics of students in the two groups were similar when the study began; hence, any subsequent differences in outcomes can be attributed to the program. The study is estimating the value added of Opening Doors, above and beyond what students normally receive. Kingsborough offers a rich array of academic programs and services, so the bar is set relatively high for the program to surpass. Also, the study examines whether the package of reforms and enhancements in Opening Doors at Kingsborough led to different outcomes, compared with standard classes and services. The study will not, however, disentangle the effects of each component.
Did the Program Make a Difference?
This report discusses the program’s impacts on a range of educational outcomes. The learning communities program directly affected students during their first semester at Kingsborough. Many higher education experts believe that students’ academic and social experiences during that first semester play a substantial role in their future success — that students who develop strong initial connections with the material they study, with other students, and with faculty are more likely to persist in college than students who do not. Also, at Kingsborough, students who make better progress in meeting their developmental English requirements may be more motivated to stay in school.
This report presents impacts for the full research sample at Kingsborough (1,534 students) for up to two years after students entered the study. The key impact findings follow.
- The program improved students’ experiences in college.
When surveyed approximately a year after entering the study, students in the program group reported that they felt more integrated at school and were more engaged with their coursework, instructors, and fellow students and had a stronger sense of belonging than did control group students. They were more likely to say that their courses required critical thinking and that they had acquired valuable academic and work skills. Finally, they were more likely to rate their college experience as “good” or “excellent.” These findings strongly suggest that the learning community program provided a markedly different experience for students. These results are similar to findings from some past studies of learning communities.
- The program improved several educational outcomes for students during the semester that they participated in the learning community.
MDRC examined students’ academic performance by using transcript data from Kingsborough. Newly enrolling students were randomly assigned for the study just before the start of their first semester in college (enrollment occurred in four different but contiguous semesters: fall 2003, spring 2004, fall 2004, and spring 2005). The first semester that each student was in the study is called the “program semester.”
Figure ES.1 illustrates some key outcomes during the program semester. The solid bars show the average outcomes for program group members, and the white bars show the averages for control group members. The difference between each pair of bars represents the program impact, if any, and asterisks indicate whether an impact is “statistically significant,” meaning that it is unlikely to be due to chance. (All impacts discussed in the Executive Summary are statistically significant.) As the two sets of bars at the left of the figure show, students in the program group attempted and passed about half a course more at Kingsborough during their first semester in the study than control group students did. They also earned almost one more “developmental credit” (called an “equated credit” at Kingsborough). Developmental courses do not earn college credit, but they do count in determining whether a student is attending school full time. Program group members were also more likely to pass all their courses during the first semester (not shown).
- The positive effects on educational outcomes diminished in later semesters of the two-year follow-up period.
The program generated a small increase in the number of credits attempted and earned in the first postprogram semester, but the effects dissipated later. By the end of the two-year follow-up period, program group members earned an average of two and a half credits more at Kingsborough than control group members. (This gain is primarily due to impacts during the program semester and, to some extent, during the first postprogram semester.)
- Opening Doors Learning Communities helped students move more quickly through the college’s developmental English requirements.
A goal of Kingsborough’s program was to help students more quickly complete developmental requirements and progress to college-level English. To enroll in the college-level course at Kingsborough, students must first pass the CUNY reading and writing skills assessment tests. (Students must pass the reading, writing, and math skills assessment tests in order to transfer to a four-year CUNY institution.) Students take the skills tests prior to enrolling; if they do not pass the reading and writing tests, they typically can retake them only after they pass specific developmental English courses.
Figure ES.2 shows the proportion of the two research groups who took the tests during their first three semesters in the study and passed the tests by the end of that period (including students who passed the tests before starting their freshman year). The program increased the proportion of students who attempted and passed the tests. Although not illustrated in the figure, most of these impacts are driven by effects in the first (program) semester. It is notable, however, that the control group members had not “caught up” in terms of test taking and passing by the end of the follow-up period.
MDRC also examined progression through English courses for different subgroups of the research sample. Among the subset of the sample who failed both English skills assessment tests before starting their freshman year, program group members were more likely than their control group counterparts to enroll in developmental English during their first two semesters. Program group members who failed one of the tests before entering college were also more likely to enroll in developmental English during their first semester and were more likely to enroll in and pass college-level English during their first two semesters. The program did not affect progression through English courses among students who had passed both English assessment tests before starting their freshman year.
- So far, the evidence is mixed about whether Kingsborough’s program increases student persistence in college.
A central goal of all the Opening Doors programs is to increase persistence in college. Initially, Kingsborough’s program did not change the rate at which students re-enrolled. In the last semester of the follow-up period, however, a difference emerged: 53 percent of the program group registered for at least one course that semester at Kingsborough, compared with 48 percent of the control group. Data from the National Student Clearinghouse, which provides enrollment information at most colleges in the nation, shows a similar effect on persistence emerging that semester. MDRC plans to continue tracking outcomes to see if this effect remains.
What Are the Implications of the Results?
Opening Doors Learning Communities at Kingsborough substantially improved students’ experiences in college and some key educational outcomes while they were in the program, but, for the most part, the effects did not persist. MDRC plans to track sample members’ outcomes for at least three years after their random assignment to the study to determine the longer-term effects on their academic performance, persistence, and graduation as well as on their later employment rates and earnings. Thus, the results in this report are not the last word on Kingsborough’s program. That said, the findings point to the following conclusions.
- Kingsborough’s learning community model shows promise as a strategy to help students move through developmental education.
Many students begin at community colleges academically underprepared for college-level work. Research shows that approximately 60 percent of freshmen beginning at community college need at least one developmental-level course.[1] Students with very low skill levels can spend a year or more in developmental courses, and many leave school before completing developmental requirements. A key challenge is how to help students meet these requirements so they can eventually complete college.
This report’s findings suggest that Kingsborough’s Opening Doors Learning Communities model is one strategy that college administrators could consider. Students in the program group were more likely to pass the English skills assessment tests — the gateway to college-level English. Also, program group students who failed one of the tests before entering school were more likely to take and pass college-level English during the follow-up period. It is important to highlight that Kingsborough’s program included English in the learning community. As a result, students in the program group were required to take English, and, as discussed above, the program substantially increased the proportion of students who took developmental English. Because students must pass developmental classes before retaking the assessment tests, this program feature is central to the impact on test taking and passing.
- The results from Kingsborough suggest that the jury is still out on whether learning communities improve students’ persistence.
As noted, Kingsborough’s program did not increase students’ retention in college until the third postprogram semester, and MDRC will collect more follow-up data to determine whether the effect continues. At this juncture, however, it is worthwhile to consider the absence, so far, of a strong program effect on retention.
Kingsborough’s program was based on the hypothesis that a more engaging and successful first semester would lead to more successful future semesters and higher rates of retention. One may wonder, however, how much change in college attendance is reasonable to expect from a one-semester program. Individuals make life choices, including whether or not to remain in college, based on myriad factors — many outside the college environment.
Even if Kingsborough’s program does not lead to substantial retention effects, it could still generate increases in degree receipt, transfer, and other longer-term outcomes, since the program group students who were still enrolled at the end of the report’s follow-up period are somewhat further along in school than the control group members.
- Enhanced services that last longer than one semester may have a more substantial effect on students.
Kingsborough’s program lasted one semester. The college’s administrators decided that there was no practical way to maintain the linked-course structure after the first semester, since students needed and wanted to take a variety of different courses in subsequent semesters. Also, the program was designed on the assumption that students’ early experiences at college influence their later success, and administrators believed that students should transition into the regular college community as quickly as possible.
The question of how long a learning community program should continue is complicated. That said, the results from the Kingsborough study suggest that participating in a learning community program for more than one semester may yield more substantial effects — the positive effects on academic outcomes were by far the largest during the first semester. If the options of a multiple-semester learning community or participating in a different learning community after the first semester are not possible, colleges could offer other kinds of enhanced services in later semesters, such as intensive counseling or more financial support.
It is worth noting that, in some of the other sites in the Opening Doors demonstration, the early results follow a similar pattern: Effects are largest when students receive enhanced services, and they diminish or even disappear after the services end.
* * *
The study at Kingsborough is of a specific program model, targeted to a certain group of students, in a particular setting. Other learning community models, target groups, and institutional settings may well lead to different results. Another rigorous study, the Learning Communities demonstration, was launched in 2006 and is using random assignment to test the effects of learning communities in up to six colleges or universities. The demonstration is part of the National Center for Postsecondary Research, funded by the U.S. Department of Education.
|
