Early Findings on Program Impacts in Three Sites

| Stephen Freedman, Daniel Friedlander

This is the first report presenting impact results from the evaluation of the Job Opportunities and Basic Skills Training (JOBS) Program, a study conducted by the Manpower Demonstration Research Corporation (MDRC) under contract to the U.S. Department of Health and Human Services with additional support from the U.S. Department of Education. It presents early findings on program impacts for three of the seven sites included in the evaluation Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. These findings are based on survey data from an early sample of people at the three sites who were required to participate in JOBS and thus became part of the research sample. The survey was conducted approximately two years after these people entered the study.

Authorized by the Family Support Act of 1988, the JOBS program was designed to provide job searchassistance, education, work experience, vocational training, and other employment-related services to recipients of Aid to Families with Dependent Children (AFDC), the largest federal-state cash welfare program. These services, however, were tied to a requirement. To the extent that state resources permitted, all single-parent AFDC recipients with children aged 3 and over (or, at state option, aged 1 and over) and at least one adult in two-parent families would have to participate in JOBS activities or face a reduction in their AFDC grant. The JOBS Evaluation was designed to determine the effect of selected types of JOBS programs on welfare recipients’ future employment and earnings, receipt of AFDC and other transfer payments, and total income, and to assess the program’s effects on the cognitive and social development, health, and school performance of recipients’ children.

Since the JOBS legislation allows for considerable flexibility, and since there was substantial uncertainty about what strategies would work best for which groups on welfare, a central objective of the evaluation was to determine the relative effectiveness of different JOBS approaches. In most evaluations, such questions are addressed by comparing results across sites implementing distinctive strategies. Although such comparisons can be very informative, questions usually remain as to whether differences in measured success result from variations in programmatic approaches across sites or from differences in local site conditions (e.g., in welfare populations, AFDC grant levels, or local labor markets). To avoid this uncertainty, the JOBS Evaluation took an innovative, largely unprecedented, and unusually rigorous approach to answering this question. The three sites that form the basis of this report agreed (for the period of this evaluation) to simultaneously operate two distinct versions of their JOBS program. The first version, which the evaluation characterizes as the “Labor Force Attachment” (LFA) approach, emphasizes rapid job entry. It relies on job search assistance, followed by work experience or short-term education or training activities for some of those who did not find jobs. The second version, characterized as the “Human Capital Development” (HCD) approach, allows job entry to be postponed so that enrollees can engage in longer, skill-building education and training activities. The intent is to increase their earning power and, hence, their long-term ability to get off and remain off AFDC. JOBS programs around the country tend to emphasize one or the other of these broad directions. The distinction between the two approaches was sharpened in these three evaluation sites to permit a side-by-side test of the two strategies’relative effectiveness.

In the three sites, AFDC applicants and recipients who were required to participate in JOBS were randomly assigned either to the LFA program group (also referred to as “LFAs”) or an LFA control group, or to the HCD program group (also referred to as “HCDs”) or an HCD control group. Random assignment assures that there are no systematic differences between the measured and unmeasured background characteristics of people in the program or control groups when they enter the study, and that, as a result, any differences in their subsequent employment and welfare experiences can be attributed with confidence to the effect of the two JOBS strategies. The term program “impacts” refers to these subsequent differences, and, in impact analyses, terms like “increases” (or “decreases”) refer to the average outcomes for a program group relative to those for a control group. Members of the LFA and HCD program groups were subject to the JOBS participation mandate and could lose part of their welfare grant if they failed to comply with JOBS’ rules (i.e., they could be “sanctioned”). Members of the LFA and HCD control groups were not subject to the JOBS participation mandate and could not receive JOBS services, but could participate in non-JOBS-provided services in their communities. The separate impacts of each of the two JOBS approaches are estimated by comparing the behavior of LFAs or HCDs to that of their respective control group.

This report is based on telephone and in-person surveys of 2,604 people in the three sites. Impact estimates have been derived from data pooled across these sites, with each site given equal weight. This sample is large enough, and the follow-up period long enough, to provide an early look at some (but not all) of the key issues in the evaluation.

The report addresses four questions: To what extent did the two types of JOBS programs in these three sites (1) increase participation in employment-directed activity, (2) increase attainment of education credentials, (3) decrease receipt of AFDC and other transfer payments, and (4) increase employment, earnings, and income? The impact results for the group assigned to the LFA version of JOBS are telling because the LFA approach is expected to produce impacts within a year or two after enrollment in the program. The reported results for the HCD version are more preliminary. Because this approach generally requires up-front investments in education and training before the search for employment begins, longer-term data may be required to capture even its initial effects on employment and earnings.

The preliminary results from the three sites show:

  • Successful implementation of the study design.

The three sites were successful in simultaneously operating two distinct, well-run, and highly mandatory LFA and HCD versions of JOBS and in carrying out the research design. Thus, the study is a fair test of these two JOBS strategies.

  • The nature of the JOBS approaches.

The LFA approach dramatically increased participation in job search activities, slightly increased the rate of participation in other work-directed services, and resulted in a high rate of sanctioning. The HCD approach also resulted in a high rate of sanctioning, but the additional services were mostly basic education Adult Basic Education, preparation for the General Educational Development (GED) test, and instruction in English as a Second Language and job search. There was also some increased participation in occupational skills training, but the HCD study tested primarily basic education and job search, not occupational training.

  • Findings for the JOBS Labor Force Attachment approach.

The LFA approach succeeded in substantially increasing the number of people who found work and left welfare within two years. Impacts on AFDC receipt, AFDC payments, employment rates, and earnings were all relatively large. Importantly, these findings resulted from impacts in all three of the study sites. The reduction in AFDC payments was very impressive overall and for all the subgroups studied for this report, and exceeded savings found in prior studies of large-scale welfare-to-work programs. Earnings gains were particularly large for people who had graduated from high school or had a GED prior to entering JOBS and for women with preschool-age children.

  • Findings for the JOBS Human Capital Development approach.

. The HCD approach through its use of basic education led to a modest increase in GED or high school diploma receipt. Within the short follow-up period for which data are currently available, however, this had not yet translated into any earnings impacts. There were AFDC reductions, although smaller than those found for the LFA approach. Impacts on AFDC payments were similar for different groups of people in JOBS, and no groups showed significant earnings impacts.

These results must be considered preliminary and subject to possible later revision, for two reasons. First, survey data were available for only 39 percent of the full survey sample in these three sites. Consequently, sample sizes are small, and the estimated impacts reflect the experience of the early-enrolling sample. Second, people were followed for only two years, which may stack the deck against the HCD approach. Future reports on findings in these three sites will use administrative records for a much larger sample, combined with the full survey, to track behavior for up to five years in order to capture more fully any long-term impacts from the education and training services and to see how long the LFA impacts last.