This report assesses the implementation and two-year impacts of two approaches to providing education and training services to employed welfare recipients in Riverside County, California. The two approaches, called Work Plus and Training Focused, together known as Phase 2, enrolled recipients of Temporary Assistance for Needy Families (TANF) benefits (primarily single parents) who worked for 20 or more hours per week but earned too little to leave assistance. Both approaches, still in operation in Riverside, encourage working TANF recipients to attend courses in remedial education, postsecondary education, or vocational training, depending on recipients’ levels of educational attainment and career aspirations. The Work Plus and Training Focused approaches offer a different mix of services, participation requirements, and messages but share the same operating principle: that, to advance in the labor market, low-wage workers need to attain skills and credentials beyond what they can acquire on the job.

To better understand the effects of encouraging employed TANF recipients to combine work with education or training, Riverside’s Work Plus and Training Focused approaches are being compared with a third, limited-services approach, called Work Focused (and referred to in this evaluation as the “control group”). Similar to postemployment programs run by states and localities (including Riverside) in the mid- to late 1990s, the Work Focused approach makes available, upon request, case management services to promote job retention and payments to defray enrollees’ child care, transportation, and other work-related expenses.

This study is part of the Employment Retention and Advancement (ERA) project, which is testing 16 models across the country. The ERA project was conceived and funded by the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS) and is also supported by the U.S. Department of Labor (DOL). The project is being conducted by MDRC, a nonprofit, nonpartisan research organization, under contract to HHS.

The findings for the Work Plus and Training Focused approaches are of particular interest because the strategy the approaches use — encouraging enrollees to combine work with education or training — stands in stark contrast to “work-based” strategies that focus almost exclusively on helping low-wage workers (1) address barriers to employment retention, such as child care and transportation problems; (2) get more hours of work, better work schedules, raises, or promotions; or (3) find a better job. The setting of the study in California is also important. In any given quarter, more than 30 percent of California’s adult TANF recipients combine work and welfare (more than 40 percent in Riverside), reflecting the state’s relatively high grant levels and rules that disregard most of recipients’ earnings when calculating their grant amounts. In a state like California, therefore, postemployment strategies must promote career advancement to help recipients earn enough to leave assistance.

History of Postemployment Programs in Riverside County

Since the mid-1980s, Riverside’s Department of Public Social Services (DPSS) has operated a mandatory preemployment program for welfare recipients, called Greater Avenues for Independence (GAIN), which excels at moving recipients quickly into jobs. However, evaluations of Riverside’s program, along with DPSS’s internal reviews, showed that many enrollees who found jobs through GAIN subsequently left employment and that relatively few advanced to better jobs. Starting in 1994, DPSS sought to address these problems by adding a postemployment component to GAIN. At first, DPSS focused primarily on providing case management services and payments for enrollees’ child care, transportation, and other work-related expenses (similar to the Work Focused approach in the present study). Evaluated as part of the national Post-Employment Services Demonstration (PESD), Riverside’s initial postemployment program did not improve enrollees’ ability to retain employment or increase their earnings beyond what they would have been without the program.

In 1998, DPSS switched to a postemployment program that encouraged enrollees to attend education or training courses while continuing to meet the state’s work requirements. DPSS named its new program Phase 2 and renamed its preemployment program Phase 1. That same year, DPSS administrators designed an alternative education- and training-focused postemployment program for TANF recipients, called New Visions. Operated by Riverside Community College, New Visions offered a flexible schedule of classes, self-paced curriculum, and short (six-week) class segments. An evaluation of New Visions found that the program did not increase employment and earnings above the levels attained by enrollees in the regular Phase 2 program.

During the ensuing years, DPSS worked with area education and training providers to make attendance in Phase 2 easier for working parents (creating courses with flexible schedules, for example) and to recruit enrollees more aggressively. The result was the version of Phase 2 referred to as the Work Plus approach. Concerned that it was difficult to combine work with education and training, DPSS administrators subsequently contracted with the Economic Development Agency (EDA) of Riverside County, the county’s Workforce Investment Agency, to design a new Phase 2 approach that encourages enrollees to maximize their hours of attendance in education or training activities, even if enrollees cut back on their hours of work. DPSS and EDA administrators named this alternative version of Phase 2 the Training Focused approach. After successfully completing a short pilot phase, EDA began operating the Training Focused approach on a countywide basis in January 2001.

Key Features of Each Approach

Table ES.1 summarizes the key features of the Work Plus, Training Focused, and Work Focused approaches.

Administration, Case Management, and Recruitment

The Work Plus approach is operated by the Phase 2 (postemployment) unit within DPSS, while the Training Focused approach is operated by the Welfare-to-Work Division within EDA. Case managers in each approach are specialists, having no enrollees in other programs in their caseload, and they actively recruit eligible TANF recipients for program services. The Work Focused approach is operated by the Phase 1 (preemployment) unit within DPSS. Work Focused enrollees are added to the caseloads of Phase 1 case managers, whose main task is to help unemployed recipients find a job. Work Focused case managers do not actively recruit eligible TANF recipients for program services.

Balance of Work and Training

Enrollees in all three approaches are subject to California’s statewide TANF rule, which requires recipients to work or engage in approved employment preparation activities for a total of 32 hours per week. Work Plus enrollees may meet this requirement with a combination of work and attendance in approved education or training activities, but they must maintain at least 20 hours of employment per week. Training Focused enrollees may substitute additional hours in school or training for hours on the job or, with their case manager’s approval, even forgo employment temporarily to participate full time in education or training activities. Work Focused enrollees are expected to meet the 32-hour requirement with at least 20 hours per week of employment, supplemented, where necessary, with participation in approved job search activities.

Education and Training Services and Philosophy

Work Plus case managers work with new enrollees to develop an Employability Plan and choose an appropriate course of study. Work Plus enrollees are responsible for contacting providers and signing up for a specific education or training program. Case managers encourage high school graduates and General Educational Development (GED) certificate recipients to attend short-term vocational training. Nongraduates are encouraged to attend classes in adult basic education or GED preparation before enrolling in vocational training.

Training Focused case managers refer new enrollees to a formal vocational assessment. After reviewing assessment results, case managers refer enrollees to specific education or training providers within EDA’s service delivery network. Case managers encourage enrollees to attend long-term vocational training courses (of up to two years duration). Nongraduates are encouraged to attend programs that combine basic education or GED preparation and vocational training.

Work Focused case managers monitor the employment status of enrollees, contact enrollees periodically, and encourage them to maximize their hours of work. Case managers do not encourage attendance in education or training activities. If requested by enrollees, Work Focused case managers provide child care and transportation payments for self-initiated education or training activities that are short term and that teach a job skill known to be in demand in Riverside County.

Income Supports

Enrollees in all three approaches are equally eligible to receive TANF benefits, food stamps, child care and transportation assistance, and medical coverage.

Response to Job Loss

Work Plus and Work Focused enrollees who leave employment remain eligible for services for up to 60 days, after which they return to DPSS’s Phase 1 (preemployment) program. Training Focused enrollees who leave employment remain eligible for services indefinitely but are required to meet California’s weekly participation mandate through participation in job search, education, or training activities.

Services for TANF Leavers

Through September 2002, enrollees in all three approaches who left TANF lost eligibility for services but could, on their own initiative, enroll in a limited-services, post-TANF program operated by Phase 1 case managers. Starting in October 2002, Work Plus and Training Focused enrollees retained their eligibility for services, but Work Focused enrollees did not.

Evaluation Design

MDRC and its subcontractor, The Lewin Group, are conducting a rigorous experimental comparison of the Work Plus and Training Focused approaches with the Work Focused approach. The study focuses on TANF recipients who became newly eligible for Phase 2 services, having enrolled in Phase 1, DPSS’s mandatory preemployment program, and having recently started a job providing 20 or more hours per week of work with wages of $6.75 or more per hour, the state minimum. DPSS randomly assigned these recipients to the Work Plus or Training Focused approaches or to the Work Focused approach, hereafter referred to as the control group. DPSS assigned Work Plus and Training Focused group members to new case managers who specialized in providing their approach’s services and reassigned members of the control group to their Phase 1 case managers. Random assignment took place between January 2001 and October 2003. This report summarizes the effects of each approach for 2,907 single parents (mostly mothers) who were randomly assigned through September 2003.

The random assignment process ensured that there are no systematic differences in sample members’ characteristics, measured and unmeasured, among the three research groups. Thus, any differences that emerge when comparing employment or other outcomes between any two of these groups can be described with confidence as true effects and not the result of chance.

Evaluation Sample

In evaluations of social policy initiatives, the background characteristics and experiences of sample members often affect the types of services they receive and their subsequent labor market behavior. Most notably for this evaluation, about 56 percent of the sample members were working for 32 hours or more per week at their time of random assignment. Having met the TANF weekly participation requirement through their work hours, these sample members remained eligible to participate in Work Plus or Training Focused activities but had no obligation to do so — not even to maintain contact with their case managers. In contrast, participation was mandatory for the 44 percent of Work Plus and the Training Focused group members who worked between 20 and 32 hours per week and had not met California’s 32-hour weekly participation requirement at their time of random assignment.

Also important, at the time of random assignment, about 42 percent of sample members lacked a high school diploma or GED certificate, the minimum credentials required to enter many training courses offered at community colleges and private vocational institutions. (Some training programs enroll nongraduates, but nongraduates often need to complete courses in basic education, English as a Second Language, or GED certificate preparation before entering their preferred course of study.) Thus, the education and training options initially open to many Work Plus and Training Focused sample members were limited.

Finally, on average, sample members had two children, and two-thirds of sample members had at least one child age 5 or younger as of random assignment. Thus, many single-parent sample members needed to arrange for child care while they worked or attended education or training courses.

Key Findings on Program Implementation and Participation

To have a fair test of the Work Plus and Training Focused approaches, a relatively large proportion of Work Plus and Training Focused group members would have to attend education or training activities, and their levels of participation in those activities would have to greatly exceed the participation level of the control group. For several reasons, these benchmarks proved difficult to achieve, most notably because a higher than expected proportion of control group members attended education or training activities on their own initiative. However, as discussed below, the two approaches attained greater success in boosting participation beyond control group levels among sample members working part-time hours at random assignment, for whom participation was mandatory, and among nongraduates, who faced greater difficulties in enrolling in vocationally oriented education or training courses.

• Compared with the control group, the Work Plus and Training Focused approaches increased participation in education or training activities primarily among sample members who, as of random assignment, were working part time or were without a high school diploma or GED certificate.

As the table shows, 32 percent of control group respondents reported that they attended an education or training activity — remedial education, postsecondary education, or vocational training — on their own initiative during the first year of follow-up. (Interviews with DPSS administrators and case managers and an examination of agency records confirmed that Phase 1 case managers almost never referred control group members to education or training activities.) Control group members’ level of participation in education or training is surprisingly high, especially for working single parents. Most likely, the setting of the evaluation in Riverside County contributed to this result. Riverside has a large number of public and private educational institutions that enroll unemployed and low-wage workers and offer Pell Grants and other support for attendees. Moreover, sample members in all three research groups enrolled and participated in essentially the same types of education and training programs, especially in the medical field (Certified Nurse’s Aide programs, in particular) and the office assistant field.

During the first year of follow-up, the Training Focused approach led to a modest increase in participation in education and training activities. About 41 percent of survey respondents in the Training Focused group attended an education or training course, 9 percentage points more than did survey respondents in the control group. Compared with the Training Focused group, a slightly smaller percentage of the Work Plus group participated in education or training (37 percent), and the difference between the Work Plus and control group rates of participation is not statistically significant. In all three groups, only about 10 to 13 percent of sample members were still participating in an education or training activity at the end of Year 1, and a similar percentage attained a degree or vocational certificate by that time (results not shown).

Results for subgroups reveal a more complex pattern (Table ES.2). Similar to the full sample, the two approaches had little or no effect on participation in education or training activities among high school graduates and GED certificate recipients or among those who were working full time (32 or more hours per week) as of random assignment. In contrast, among those who were working part time (20 to 30 hours per week), around half of Work Plus and Training Focused respondents participated in an education or training activity, compared with only about one-third of the control group. The differences of 18 percentage points for the Work Plus group and 14 percentage points for the Training Focused group are statistically significant and represent relatively large effects on participation. Among nongraduates, a similarly large impact on (of 17 percentage points) was found for Training Focused group members. A higher percentage of Work Plus group members participated in an education or training activity as well, but the difference is smaller (10 percentage points) and not statistically significant.

Several factors appear to explain why the Work Plus and Training Focused approaches led to only modest increases in participation in education or training activities beyond what control group members attained on their own initiative. Work Plus and Training Focused case managers reported difficulty in convincing many employed single parents, especially those working full time, to cut back on their hours of work or on time devoted to family in order to attend school or training. Other sample members stopped participating or chose not to enroll in education or training programs when they left employment — opting, instead, to look for work. Additionally, about half of the sample left TANF within one year of random assignment (results not shown), which, during much of this report’s follow-up period, ended their eligibility for services. Finally, for part of the follow-up, a shortage of funds that were intended to pay for specialized training opportunities reduced the number of openings in longer-term training programs, especially for members of the Training Focused group.

Findings on Employment and Earnings Impacts

• Over the two-year follow-up period, neither the Work Plus nor the Training Focused approach led to greater employment retention rates or higher earnings than the levels achieved by the control group.

For each research group, Table ES.3 displays average quarterly employment rates (a measure of employment retention, showing the percentage of follow-up quarters with employment) and total earnings during Years 1 and 2. These measures were calculated from quarterly earnings reported to California’s unemployment insurance (UI) system. All sample members were working as of random assignment; therefore, employment levels could only move downward over time. For instance, control group members worked at UI-covered jobs during 72 percent of the follow-up quarters in Year 1 but during only 62 percent of the quarters in the following year — indicating a fairly rapid decline in employment. Control group members earned relatively little during the follow-up period, averaging about $8,350 per year in total earnings (including zero earnings for group members without employment in a UI-covered job).

In keeping with their advancement goals, the Work Plus and Training Focused approaches are expected to increase employment retention and total earnings above control group levels. However, education and training initiatives generally do not lead to employment and earnings impacts in the first year of follow-up, while most participants are attending school or training.

To date, the results for each approach are not encouraging. During the first two years of follow-up, Work Plus and Training Focused group members remained employed at UI-covered jobs for about the same number of quarters as control group members and received, on average, about the same amount in total earnings. The only statistically significant impact recorded during the follow-up period is for the Training Focused group during Year 1 — a modest decrease in quarterly employment of 4.9 percentage points below the control group. This result probably reflects the choice made by some Training Focused group members to temporarily forgo employment while they attended education or training activities. The difference in quarterly employment between the Training Focused and control groups diminished during Year 2 and was no longer statistically significant.

In addition to results for the full samples, Table ES.3 displays results for graduates and nongraduates and for part-time and full-time workers. As with the full sample, the Work Plus approach did not increase employment or earnings above the control group for any subgroup. The pattern of impacts is nearly as consistent for the Training Focused group, involving a decrease in quarterly employment in Year 1 (for three of the subgroups) and no statistically significant effect on earnings during either year of follow-up.

Conclusions and Policy Implications

A two-year follow-up period may be too short to assess the impacts of education and training initiatives for working TANF recipients. However, the findings from the Riverside study so far underscore the difficulty of designing and implementing education and training initiatives for low-income adults under the conditions that governed the Riverside study. These problems made it harder for each approach to raise participation in education and training activities beyond control group levels. They include:

• Services were targeted to TANF recipients who had only recently started employment. It may be difficult to convince people who are adjusting to their new jobs to participate in activities aimed at achieving career advancement in the long term.



• Most enrollees were already working full time.



• Enrollees were expected to attend education or training courses by traveling to traditional venues like adult education schools, community colleges, or vocational training institutions during nonwork hours.



• Attendance at school or training sometimes required enrollees to decrease their income, at least temporarily, by reducing their work hours or forgoing employment.

This finding applies more to sample members who were working full time at random assignment and to high school graduates and GED certificate recipients — subgroups that exhibited little or no increase in participation in education or training beyond their counterparts in the control group — than to part-time workers and nongraduates. For the latter two groups, which represent more disadvantaged TANF populations, the Work Plus and Training Focused approaches increased attendance in skill-building activities — particularly in adult basic education or GED certificate preparation classes — but so far have not led to higher levels of employment or earnings beyond what would have happened without either intervention.

Finally, it is worth noting that the Work Plus and Training Focused approaches are only two of several advancement strategies for low-income adults that encourage attendance at school or training. Examples of other similar programs that have shown promise in previous or ongoing evaluations include: mandatory education-focused preemployment programs for Aid to Families with Dependent Children (AFDC) or TANF recipients in Atlanta, Georgia, and Columbus, Ohio (two of seven programs evaluated in the National Evaluation of Welfare-to-Work Strategies [NEWWS] that stressed education or training); and an initiative involving two community colleges in the New Orleans, Louisiana, area (part of the Opening Doors demonstration), which offers low-income parents enhanced scholarships if they remain enrolled and maintain a minimum grade point average. Other initiatives currently under study include training programs operated at the workplace and sectoral employment initiatives (involving business groups, unions, government agencies, and community-based organizations, individually or in partnership) that develop career opportunities and training curricula for low-wage workers in specific industries. In the coming years, it will be important for program administrators and policymakers to understand the long-term effects of the Work Plus and Training Focused approaches, as well as those of similar alternative approaches. There is still much to learn about which services and supports offer the greatest promise of helping low-income adults advance in the labor market.