This is the first report presenting impact results from the evaluation of the Job Opportunities and Basic Skills Training (JOBS) Program, a study conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services with additional support from the U.S. Department of Education. It presents early findings on program impacts for three of the seven sites included in the evaluation Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. These findings are based on survey data from an early sample of people at the three sites who were required to participate in JOBS and thus became part of the research sample. The survey was conducted approximately two years after these people entered the study.

Authorized by the Family Support Act of 1988, the JOBS program was designed to provide job searchassistance, education, work experience, vocational training, and other employment-related services to recipients of Aid to Families with Dependent Children (AFDC), the largest federal-state cash welfare program. These services, however, were tied to a requirement. To the extent that state resources permitted, all single-parent AFDC recipients with children aged 3 and over (or, at state option, aged 1 and over) and at least one adult in two-parent families would have to participate in JOBS activities or face a reduction in their AFDC grant. The JOBS Evaluation was designed to determine the effect of selected types of JOBS programs on welfare recipients' future employment and earnings, receipt of AFDC and other transfer payments, and total income, and to assess the program's effects on the cognitive and social development, health, and school performance of recipients' children.

Since the JOBS legislation allows for considerable flexibility, and since there was substantial uncertainty about what strategies would work best for which groups on welfare, a central objective of the evaluation was to determine the relative effectiveness of different JOBS approaches. In most evaluations, such questions are addressed by comparing results across sites implementing distinctive strategies. Although such comparisons can be very informative, questions usually remain as to whether differences in measured success result from variations in programmatic approaches across sites or from differences in local site conditions (e.g., in welfare populations, AFDC grant levels, or local labor markets). To avoid this uncertainty, the JOBS Evaluation took an innovative, largely unprecedented, and unusually rigorous approach to answering this question. The three sites that form the basis of this report agreed (for the period of this evaluation) to simultaneously operate two distinct versions of their JOBS program. The first version, which the evaluation characterizes as the "Labor Force Attachment" (LFA) approach, emphasizes rapid job entry. It relies on job search assistance, followed by work experience or short-term education or training activities for some of those who did not find jobs. The second version, characterized as the "Human Capital Development" (HCD) approach, allows job entry to be postponed so that enrollees can engage in longer, skill-building education and training activities. The intent is to increase their earning power and, hence, their long-term ability to get off and remain off AFDC 1. JOBS programs around the country tend to emphasize one or the other of these broad directions. The distinction between the two approaches was sharpened in these three evaluation sites to permit a side-by-side test of the two strategies'relative effectiveness.

In the three sites, AFDC applicants and recipients who were required to participate in JOBS were randomly assigned either to the LFA program group (also referred to as "LFAs") or an LFA control group, or to the HCD program group (also referred to as "HCDs") or an HCD control group. Random assignment assures that there are no systematic differences between the measured and unmeasured background characteristics of people in the program or control groups when they enter the study, and that, as a result, any differences in their subsequent employment and welfare experiences can be attributed with confidence to the effect of the two JOBS strategies. The term program "impacts" refers to these subsequent differences, and, in impact analyses, terms like "increases" (or "decreases") refer to the average outcomes for a program group relative to those for a control group. Members of the LFA and HCD program groups were subject to the JOBS participation mandate and could lose part of their welfare grant if they failed to comply with JOBS' rules (i.e., they could be "sanctioned"). Members of the LFA and HCD control groups were not subject to the JOBS participation mandate and could not receive JOBS services, but could participate in non-JOBS-provided services in their communities. The separate impacts of each of the two JOBS approaches are estimated by comparing the behavior of LFAs or HCDs to that of their respective control group. 2

This report is based on telephone and in-person surveys of 2,604 people in the three sites. Impact estimates have been derived from data pooled across these sites, with each site given equal weight. This sample is large enough, and the follow-up period long enough, to provide an early look at some (but not all) of the key issues in the evaluation.

The report addresses four questions: To what extent did the two types of JOBS programs in these three sites (1) increase participation in employment-directed activity, (2) increase attainment of education credentials, (3) decrease receipt of AFDC and other transfer payments, and (4) increase employment, earnings, and income? The impact results for the group assigned to the LFA version of JOBS are telling because the LFA approach is expected to produce impacts within a year or two after enrollment in the program. The reported results for the HCD version are more preliminary. Because this approach generally requires up-front investments in education and training before the search for employment begins, longer-term data may be required to capture even its initial effects on employment and earnings.

The preliminary results from the three sites show:

• Successful implementation of the study design.

The three sites were successful in simultaneously operating two distinct, well-run, and highly mandatory LFA and HCD versions of JOBS and in carrying out the research design. Thus, the study is a fair test of these two JOBS strategies.

• The nature of the JOBS approaches.

The LFA approach dramatically increased participation in job search activities, slightly increased the rate of participation in other work-directed services, and resulted in a high rate of sanctioning. The HCD approach also resulted in a high rate of sanctioning, but the additional services were mostly basic education Adult Basic Education, preparation for the General Educational Development (GED) test, and instruction in English as a Second Language and job search. There was also some increased participation in occupational skills training, but the HCD study tested primarily basic education and job search, not occupational training. 3

• Findings for the JOBS Labor Force Attachment approach.

The LFA approach succeeded in substantially increasing the number of people who found work and left welfare within two years. Impacts on AFDC receipt, AFDC payments, employment rates, and earnings were all relatively large. Importantly, these findings resulted from impacts in all three of the study sites. The reduction in AFDC payments was very impressive overall and for all the subgroups studied for this report, and exceeded savings found in prior studies of large-scale welfare-to-work programs. Earnings gains were particularly large for people who had graduated from high school or had a GED prior to entering JOBS and for women with preschool-age children.

• Findings for the JOBS Human Capital Development approach.

. The HCD approach through its use of basic education led to a modest increase in GED or high school diploma receipt. Within the short follow-up period for which data are currently available, however, this had not yet translated into any earnings impacts. There were AFDC reductions, although smaller than those found for the LFA approach. Impacts on AFDC payments were similar for different groups of people in JOBS, and no groups showed significant earnings impacts.

These results must be considered preliminary and subject to possible later revision, for two reasons. First, survey data were available for only 39 percent of the full survey sample in these three sites. Consequently, sample sizes are small, and the estimated impacts reflect the experience of the early-enrolling sample. Second, people were followed for only two years, which may stack the deck against the HCD approach. Future reports on findings in these three sites will use administrative records for a much larger sample, combined with the full survey, to track behavior for up to five years in order to capture more fully any long-term impacts from the education and training services and to see how long the LFA impacts last.

Findings for the Labor Force Attachment JOBS Approach

This section presents the findings for the LFA approach; the next section presents the findings for the HCD approach.

• What is the Labor Force Attachment JOBS approach?

Staff implementing the LFA approach used job search; short-term education or training; and other services to implement the JOBS participation mandate and to move people quickly into employment. Since people in the control group could also access similar services on their own (i.e., as voluntary participants in non-JOBS-provided services), a key initial question is whether JOBS' mandate actually increased the rate at which people were active in employment-directed services above and beyond the activity rate of the control group. 4 Overall, according to the survey, 55 percent of LFAs who showed up at a JOBS orientation participated in an employment-directed activity, 5 compared to 34 percent of LFA control group members. The increase in activity for LFAs compared to LFA control group members was largely the result of LFAs' participation in job search activities. There was also a small increase in participation in basic education and work experience or on-the-job training. (Although about half of LFAs' participation was in education or training, a similar percentage of LFA control group members also participated in these activities on their own initiative, resulting in virtually no participation difference between the LFAs and the LFA control group.) It is possible, however, that LFAs participated in different types of education and training, or for different lengths of time, than control group members.

• Did the Labor Force Attachment approach result in a subsequent change in welfare receipt and payments?

At the end of the two-year follow-up period, LFAs were substantially less likely to be on AFDC than members of the LFA control group. As shown in Table 1, 68 percent of LFA control group members reported AFDC income during the final month prior to the survey interview (i.e., the 25th month, on average, counting the month of random assignment as month one). The receipt rate among LFAs was 57 percent, yielding an 11 percentage point reduction in the percent on AFDC. The average dollar amount of AFDC payments for the month was reduced from $276 for LFA control group members to $216 for LFAs, for a saving of $61 per month, or 22 percent relative to the LFA control group level. (The AFDC payment averages shown in the table include zeroes for sample members who were not receiving AFDC.) This percentage reduction exceeds that found in prior similar studies of large-scale welfare-to-work programs.

Most of the AFDC savings appear to be associated with LFAs' having left welfare rather than with lower grants for LFAs who were still on the rolls. Sanctions contributed to these savings: About one-fifth of LFAs reported having been sanctioned during the two-year survey follow-up period a rate that points to tougher enforcement of program rules than was found in prior studies of JOBS and pre-JOBS welfare-to-work programs. Food Stamp payments during the final month were also reduced, by 14 percent, because fewer LFAs were receiving Food Stamps at that point. Based on the final month's impacts, the combined annualized savings for AFDC and Food Stamps would be $984 per LFA program group member.

• Did the Labor Force Attachment approach result in a subsequent change in employment and earnings?

Being in the LFA program group also led to relatively large impacts on employment and earnings. During the final month before the survey interview, 42.5 percent of LFAs said they were employed (see the first row of Table 1), an increase of 8.1 percentage points over the 34.4 percent for LFA control group members. Full-time employment increased (not shown), and about half of the employment impact (3.8 percentage points) came from a higher percentage of LFAs than control group members getting jobs that paid $10,000 per year or more. As a consequence of the increased employment, average earnings for the month increased from $226 for LFA control group members to $285 for LFAs, an impact of 26 percent relative to the LFA control group level. (The averages for earnings include zero values for people who were not employed.)

• Did impacts vary for people with different characteristics?

The LFA approach was also successful with the several subgroups analyzed separately for this report. Earnings impacts were particularly large for those who had a high school diploma or GED certificate when they entered the JOBS study, and for mothers of preschool-age children. Future reports will examine the extent to which mothers of young children used JOBS child care, and the cost-effectiveness of the program for this group. The AFDC savings were similar for both of these groups, as well as for mothers of older children and for people who did not have a diploma or GED when they entered the research sample.

• What were some of the other impacts?

Table 1 shows that fewer LFAs than LFA control group members were covered by Medicaid or private health insurance. In addition, since earnings gains were about the same size as AFDC reductions, the LFA approach produced no significant change in measured income.

• Were the impacts concentrated in any particular site?

The available sample is too small to provide reliable impact estimates for each of the three sites. However, the question arises as to whether one site may have dominated the pooled impact estimates. This is important because Riverside County, in a previous evaluation of California's Greater Avenues for Independence (GAIN) Program, produced what were the largest impacts on earnings and AFDC payments for a large-scale, experimentally evaluated program up to that date. In the present study, no one site dominated the pooled impact estimate. The early data used in this report suggest that Grand Rapids had the largest impact on average earnings, followed by Riverside, with Atlanta third. Dropping Riverside from the pooled estimates would have left the average earnings impact virtually unchanged. The sites ranked in the same order on welfare savings (in percentage terms), but Riverside had the largest average dollar savings because of California's relatively high monthly AFDC benefit. Thus, Riverside did make a somewhat greater-than-proportionate contribution to the pooled AFDC impact: Savings for LFAs would have been about one-third lower without Riverside.

Findings for the Human Capital Development JOBS Approach

• What is the Human Capital Development JOBS approach?

Staff implementing the HCD approach, in contrast to those implementing the LFA approach, encouraged people to postpone taking a job, in the expectation that their earning power would be increased and their long-term welfare dependency reduced if they participated in skill-building activities. The JOBS legislation had explicitly encouraged these longer-term investments because prior research had pointed to the limitations of job-search-only programs in producing major changes and succeeding with long-term recipients.

Since people in the HCD control group could, on their own, enroll in education, training, and other services that were widely available in the community, a key initial question is whether the JOBS participation mandate led to any greater use of these services. The survey data show that, overall, 64 percent of HCDs who attended a JOBS orientation participated in an employment-directed activity, nearly double the 33 percent participation rate for HCD control group members. Most of this difference was accounted for by increased participation in Adult Basic Education and GED classes. HCDs also participated in job search activities more frequently. There was a much smaller difference in participation in occupational training, and there was no increase in enrollment in post-secondary education. As a consequence of the increased education activity, 7 percent of HCDs reported obtaining a GED or high school diploma in the two-year period following enrollment in the study, more than 4 percentage points above the rate for HCD control group members.

• Is two years long enough for assessing the impacts of the Human Capital Development approach?

Education and skills training programs are usually more expensive than job search assistance, and they may increase AFDC costs initially by keeping some participants on the rolls longer than they would have remained otherwise. Advocates of such approaches hope that they will be more effective in the long run in moving JOBS enrollees who are not initially job-ready into the work force and off AFDC, and that these approaches will decrease the probability that people subsequently lose their jobs and return to AFDC. Those effects, however, may take several years to appear. Thus, it is likely that the two-year follow-up available for this report is not long enough to capture the full or possibly even the initial effects of lengthy basic education or training activities. This points to the importance of future reports to clarifying the relative effectiveness of these two JOBS strategies.

• Did the Human Capital Development approach result in a subsequent change in welfare receipt and payments?

As shown in Table 2, there were significant AFDC savings for HCDs, although not as large as for LFAs. In the final month before the survey interview, receipt of AFDC was down from 69 percent for the HCD control group to 65 percent for HCDs. Average AFDC payments were down by $38 per month, or 14 percent, relative to the HCD control group. While this reduction is less than the 22 percent produced by the LFA approach, it is large, and at the high end of achievements from welfare-to-work programs evaluated during the 1980s, which provided primarily job search assistance. For HCDs, reductions in average payments appear to be the result of two factors: fewer HCD's receiving welfare in the final month and lower grant averages for those still on AFDC. Also, one in four reported having received a sanction during the two-year survey follow-up period, again reflecting tough enforcement of the program's mandate. There were no significant impacts on Food Stamp receipt.

• Did the Human Capital Development approach result in a subsequent change in employment and earnings or other impacts?

At the end of the two-year follow-up period, there were no overall significant impacts on the employment or earnings of HCDs. There were, however, significant impacts on full-time employment in the first year (not shown in Table 2). In addition, a separate analysis of administrative records data for a larger sample in Atlanta suggests that earnings and employment impacts may be appearing in that site toward the end of the second follow-up year.

These estimates suggest that the effects on earning power of the lengthier, more intensive HCD activities have not yet been felt by the HCD sample. It will be important, when additional follow-up data become available, to look for third-, fourth-, and fifth-year effects on employment and earnings, which may, in turn, yield additional effects on AFDC receipt and AFDC payments.

The absence of earnings gains, combined with the decline in welfare payments, resulted in a statistically significant reduction (6 percent) in measured income for HCDs. There was no significant change in the percentage of HCD families covered by health care insurance.

• Did impacts vary for people with different characteristics?

As with LFAs, impacts on AFDC payments were similar for sample members with and without a high school diploma or GED and for those who did or did not have a child under the age of 6. There were no significant earnings impacts for any of those groups. However, among high school dropouts, more HCDs than HCD control group members were working at the end of two years.

• Did one site perform better than the average?

Conclusion

This report presents early evidence that well-implemented, highly mandatory JOBS programs that use job search followed by a range of short-term education, training, and other services to promote rapid job entry the Labor Force Attachment model can produce dramatic reductions in welfare receipt and substantial increases in employment and earnings. These results provide an important indication that JOBS programs can produce large welfare savings, but at least at this stage in the study not at the cost of significantly diminishing the incomes of families (though some families lost health benefits). Future reports will examine other impacts (such as effects on children); whether the present impacts on earnings, welfare payments, and health insurance coverage continue beyond two years; and how program benefits compare to program costs

The report offers a more preliminary picture of the returns to well-implemented, highly mandatory programs that provide primarily basic education but also a range of other services (including job search assistance and long-term training) the Human Capital Development approach. Although here, too, there were notable welfare savings, there were not yet any consistent gains in employment or earnings. Future reports will examine whether earnings impacts improve over the long term, and whether impacts on labor market and welfare outcomes are larger for sites and subgroups that made greater use of occupational skills training in their strategy to improve participants' capacity to get and hold higher-wage jobs.

Notes

1 A fourth site, Columbus, Ohio is testing the relative effectiveness of two different ways of structuring case managers' functions. In the other three sites Detroit, Oklahoma City, and Portland, Oregon the existing JOBS program is being evaluated.

2 See the discussion in Chapter 1 of the full report for details on the construction of the sample used for this report.

3 Future reports will use larger samples to isolate the impact of JOBS for sites and subgroups that made greater use of occupational skills training

4 A companion report (U.S. Department of Health and Human Services and U.S. Department of Education, The JOBS Evaluation: Monthly Participation Rates in Three Sites and Factors Affecting Participation Levels in Welfare-to-Work Programs, 1995) presents a more detailed examination of JOBS participation for people required to participate in the LFA and HCD versions of the program.

5 The participation rates cited here for LFAs and HCDs are much higher than the official monthly JOBS participation rates for these programs, for two main reasons. First, the rates in this report include only those individuals who showed up at a JOBS orientation session, which is the step preliminary to actual participation in the program. Second, the rates reported here represent those who ever participated in JOBS during the two-year follow-up period.