The Personal Responsibility
and Work Opportunity Reconciliation Act of August 19961
ended the Aid to Families with Dependent
Children (AFDC) program, one of the nation’s principal safety
nets for poor families. Among its provisions, the law replaced
AFDC with a block grant program, Temporary Assistance for
Needy Families (TANF), and created financial incentives for
states to run mandatory, work-focused welfare-to-work programs.
While these types of programs are not new, various aspects
of the 1996 law increase their importance: federal funds now
may not be used to support most families on welfare for longer
than five years and a number of states and localities have
shorter welfare time limits; states face financial penalties
if they fail to meet TANF-defined “participation standards,”
which require large proportions of welfare recipients to be
in work or work-related activities; and states must have a
plan for how they will require recipients to work after two
years of assistance.
To
meet the new challenges of the federal welfare legislation,
state and local administrators and policy makers need to know
about the types of welfare-to-work program approaches that
can quickly move substantial numbers of people into work and
off welfare. This report provides such guidance, by analyzing
the effectiveness of 11 mandatory welfare-to-work programs
operated in seven locales. The sites included in the evaluation
are Atlanta, Georgia; Columbus, Ohio; Detroit and Grand Rapids,
Michigan; Oklahoma City, Oklahoma; Portland, Oregon; and Riverside,
California.
The
report is one in a series from an evaluation of the programs
called the National Evaluation of Welfare-to-Work Strategies
(NEWWS), conducted by the Manpower Demonstration Research
Corporation (MDRC) under contract to the U.S. Department of
Health and Human Services (HHS), with support from the U.S.
Department of Education. Child Trends, as a subcontractor,
is conducting the analyses of outcomes for young children
(the Child Outcomes Study). Two other recent reports (both
also published in 2000 by the U.S. Department of Health and
Human Services, Administration for Children and Families and
Office of the Assistant Secretary for Planning and Evaluation,
and the U.S. Department of Education) should be viewed as
“companion” documents to this report: Impacts on Young Children and Their Families
Two Years After Enrollment: Findings from the Child Outcomes
Study, prepared by Sharon M. McGroder, Martha J. Zaslow,
Kristin A. Moore, and Suzanne M. LeMenestrel, Child Trends;
and Do Mandatory Welfare-to-Work
Programs Affect the Well-Being of Children? A Synthesis of
Child Research Conducted as Part of the National Evaluation
of Welfare-to-Work Strategies, prepared by Gayle Hamilton,
MDRC, with Stephen Freedman, MDRC, and Sharon M. McGroder,
Child Trends.
Each
of the 11 studied programs operated under the federal Job
Opportunities and Basic Skills Training (JOBS) program, which
preceded TANF. Unlike TANF, these programs did not impose
a time limit on eligibility for welfare assistance. However,
they shared TANF’s primary goal of moving welfare recipients
into paid work and off assistance. Further, among the 11 programs
some are strongly employment-focused, the welfare-to-work
strategy favored under TANF, and some are strongly basic education-focused,
an approach possible under TANF but more prevalent during
the late 1980s and early 1990s. (Overall, the present results
pertain to the period between 1991 and 1996.) The programs
varied in other ways, including how broadly the participation
mandate was applied to the welfare caseload and how strictly
it was enforced, the amount of child care support provided
for program participation or employment, and methods of case
management. The programs also served different welfare populations
and operated in a variety of labor markets.
Taking advantage
of the array of programs studied as part of the evaluation,
this report addresses the following critical question: What
works best, and for whom? The report distinguishes between
employment-focused and basic education-focused programs, as
well as between levels of enforcement of the participation
mandate. Taking into account these two dimensions of program
characteristics, plus the types of program activities to which
welfare recipients were assigned, four categories of welfare-to-work
program approaches emerge:
·
employment-focused
programs, with first assignments made to job search and a
high level of participation mandate enforcement;
·
employment-focused
programs, with first assignments made to job search, basic
education, or vocational skills training and a high level
of participation enforcement (only one program falls into
this category);
·
education-focused
programs, with first assignment made to basic education or
skills training and a high level of participation enforcement;
and
·
education-focused
programs, with first assignments made to basic education or
skills training and a low level of participation enforcement.
Exhibit ES-1 categorizes the
sites’ programs. Notably, four of the sites (Atlanta, Grand
Rapids, Riverside, and Columbus) operated two different programs
simultaneously, to enable rigorous side-by-side tests of the
comparative effectiveness of various approaches. Three sites
implemented a Labor Force Attachment (LFA) program as well
as a Human Capital Development (HCD) program, versions of
employment-focused and education-focused programs that magnified
the differences between the two types of approaches. The fourth
site, Columbus, implemented a program using a traditional
(TRD) case management model, in which welfare eligibility
and employment program functions were performed by separate
staff members, and a program using an Integrated (INT) case
management model, in which these two functions were performed
by the same staff member. These eight programs in four sites,
described in more detail in Section II, are referred throughout
by their site name and shortened program model name (LFA,
HCD, TRD, or INT).
It
is important to note that the studies of the programs in the
education-focused category yield information about the effects
of increasing welfare recipients’ participation in basic education
programs (including Adult Basic Education, GED preparation,
and English as a Second Language classes) and, to a much lesser
extent, in vocation skills training programs, but not in college.
On their own, many welfare recipients enroll in various types
of education or training classes and reap benefits from them;
the education-focused programs in the evaluation, however,
sought to increase participation in education or training
activities beyond what would normally occur. As will be discussed
below, most of the programs did indeed increase such participation,
but the increases in enrollments were in basic education courses
and, to some degree, in vocational training courses, and not
in college-level ones.
This
report analyzes the programs’ effects for single-parent welfare
recipients, focusing on results for the two years after individuals
entered the programs. This is an important period in which
to gauge whether programs moved recipients from welfare to
work. Many states and localities now terminate welfare eligibility
after two years. In addition, prior research has shown that
many individuals on welfare for at least two years will likely
remain on the rolls for a considerably longer time. Under
TANF, these individuals would be in jeopardy of reaching their
five-year limit on federal funding for welfare benefits. Consequently,
the two-year results for these 11 programs will become a benchmark
for the next generation of welfare initiatives.
The
report explores the following questions:
·
Which welfare-to-work program
approaches were most successful in helping welfare recipients
to receive the program services or attain the skills or credentials
that could enhance their chances of finding employment?
·
Which approaches were most
successful in helping welfare recipients to find paid work
and leave welfare within the two-year follow-up period and
to remain off welfare? Did any approaches help individuals
to get a “good” job, that is, a full-time job with health
benefits?
·
Which approaches were most
successful in increasing welfare recipients’ income and helping
them move out of poverty?
·
Did any approaches positively
or negatively affect the well-being of children?
·
Which approaches were most
successful in achieving self-sufficiency for those who were
at high risk for long stays on welfare?
The
NEWWS Evaluation uses an unusually strong research design,
a random assignment experiment, to estimate program effects.
In each site individuals who were required to participate
in the program were assigned at random to either a program
group (in some sites, one of two program groups) or a control
group. Program group members had access to program-provided
services and were required to participate in the program or
risk a reduction in their monthly welfare grant. Control group
members received no mandatory welfare-to-work program services
but could seek similar services on their own in the community.
This random assignment design assures that within each site
there are no systematic differences between the background
characteristics of program and control group members when
they enter the study. In addition, within each site program
and control group members are subject to the same welfare
grant levels, labor market conditions, and other environmental
factors. As a result, any subsequent differences in outcomes
between the groups within each site can be attributed with
confidence to the effects of the program. These differences,
called impacts, can then be compared across sites, yielding
a much more accurate determination of which types of programs
are high and low performers than simple comparisons of statistics,
such as welfare caseload reductions, across localities or
states.
I.
Findings in Brief
An
examination of the range of effects achieved by all 11 programs
yielded the following information about which welfare-to-work
program strategies are more or less successful in helping
welfare recipients achieve self-sufficiency:
All programs, regardless of their approach,
increased participation in activities designed to promote
employment during the two-year follow-up period. As expected,
employment-focused programs increased participation primarily
in job search activities, whereas education-focused programs
raised participation levels primarily in basic education and
vocational skills training classes. Very different patterns
of participation impacts were found for individuals who entered
the study with a high school diploma or GED certificate and
for those who did not have these credentials. In most education-focused
programs participation impacts were concentrated among those
without a high school diploma or GED and resulted primarily
from large increases in attendance in basic education; only
small increases in attendance in post-secondary education
or vocational training were found for the education-focused
programs, and they were generally among only high school graduates
or GED holders. In contrast, large impacts on participation
in job search were achieved for both groups in the employment-focused
programs.
Some education-focused programs, as well as
the Portland program, were able to produce relatively large
impacts (about 10 percentage points) on GED attainment among
sample members who did not have a high school diploma or GED
certificate at study entry. Of the seven education-focused
programs, Grand Rapids HCD, Riverside HCD, and Columbus Traditional
programs had this effect. Portland’s program, in addition
to boosting GED receipt, increased the rate at which those
without education credentials obtained a trade license or
certificate by 12 percentage points. For sample members with
a high school diploma or GED certificate at study entry, only
three programs (Atlanta LFA and HCD and Grand Rapids HCD)
increased receipt of a trade license or certificate.
As expected, employment-focused programs produced
larger gains in employment and earnings over the two-year
follow-up period than education-focused programs, but these
effects may not be sustained everywhere in the long run.
Except in Riverside, the site with the most difficult labor
market, a majority of control group members found jobs on
their own initiative at some point within two years of random
assignment and, as a group (including zeroes for nonearners),
had average earnings during the second year of follow-up ranging
from $2,127 (Oklahoma City) to $3,978 (Columbus). In Portland
program group members attained the largest earnings increase
among all programs, averaging more than $900 per year in earnings
above control group members. Equally important, employment
and earnings gains in Portland grew larger over time and reached
their highest levels at the end of year 2, the end of the
short-term follow-up period available for this report. The
other employment-focused programs produced moderate earnings
increases, ranging from $400 to $650 per year, that grew smaller
toward the end of year 2.
Several of the education-focused programs
began to show moderate impacts in year 2. By the end of
year 2 all but two of the education-focused programs had attained
increases in employment and earnings that equaled or exceeded
the gains achieved by all employment-focused programs except
Portland’s. The two exceptions to this pattern, the Riverside
HCD and Oklahoma City programs, did not raise employment or
earnings levels in year 2. Overall, these results underscore
the importance of tracking the effects of education-focused
programs over a longer term.
All programs reduced welfare dependency to
some degree. Control group members in all but one site
remained on welfare for an average of 16 to 20 months during
the two-year follow-up period and received payments averaging
between $3,624 (Oklahoma City) and $10,302 (Riverside HCD)
during this same period. Seven of the 11 programs, a mixture
of employment-and education-focused approaches, decreased
cumulative welfare expenditures by more than 10 percent, a
historically large effect; welfare reductions in the other
four programs were smaller. Portland’s program produced a
large decrease in welfare receipt that persisted at a high
level throughout the follow-up period, showing a 12 percentage
point decrease in welfare receipt during the last quarter
of the two-year period; all other programs had reduced welfare
receipt at this point by 3 to 7 percentage points. All in
all, however, at least 40 percent of sample members in the
programs were still relying to some extent on welfare at the
end of two years.
Most programs increased sample members’ reliance
on earnings, as opposed to welfare, but family net incomes
were largely unchanged. As a result, the programs lifted few
families above the poverty line. Reductions in welfare,
Food Stamps, and other benefits generally matched or exceeded
earnings gains. Including estimates of the Earned Income Tax
Credit (EITC) as income produced little change in this finding
for all programs except Portland’s, which attained the largest
and most consistent gain in total income ($238, or $425 including
the EITC estimate, for year 2 of the follow-up) and also produced
a small increase in the proportion with incomes above the
poverty level (4 percentage points, or 7 percentage points
including the EITC estimate, in year 2).
Although no programs had pervasive negative
effects on sample members, some individuals were adversely
affected. In year 2 of follow-up six programs (some employment-focused
and some education-focused) produced small increases in the
proportion of sample members with combined income from AFDC,
Food Stamps, and earnings equivalent to less than 50 percent
of poverty levels. In addition, several programs (representing
both types of approaches) increased the rate at which individuals
left welfare without a job. Finally, some programs that increased
employment also decreased family health insurance coverage
(as reported by parents) and increased out-of-pocket child
care expenditures.
The programs did not have widespread, large,
or consistent effects on the children of sample members, but
positive and negative effects occurred in some programs.
No programs in the evaluation provided direct services (with
the exception of child care assistance) to children. Program-produced
changes in the lives of sample members (virtually all mothers)
may, nevertheless, influence the well-being of children. There
is evidence that some of the programs affected the likelihood
of at least one child in a family having behavioral, educational,
or health and safety problems. There was not, however, a consistent
pattern of benefit or harm to children. In addition, employment-
and education-focused programs did not appear to affect children
differently; there was no consistent evidence that one particular
approach affected children more or less or was more likely
to help or harm children.
Several employment- and education-focused
programs attained at least moderate employment and earnings
gains for the “most disadvantaged” sample members. Five
pro-grams (Portland, Grand Rapids LFA and HCD, and Riverside
LFA and HCD) increased employment and earnings for individuals
who at study entry did not have a high school diploma or GED,
had not worked in the prior year, and had been on welfare
cumulatively for two years or more. These five programs and
two others (Detroit and Columbus Integrated) also reduced
the amount of time that the most disadvantaged individuals
spent on welfare during the follow-up period.
High enforcement programs did not produce
the largest impacts, but low enforcement programs resulted
in only small effects. Programs in which staff closely
monitored individuals’ attendance in program activities, followed
up quickly when problems arose, and swiftly imposed financial
sanctions when individuals did not comply with program requirements,
were present among both the employment- and education-focused
programs. High enforcement programs, notably those in Grand
Rapids and Columbus, did not necessarily produce the largest
impacts. However, the two low enforcement programs — Oklahoma
City and, in its early stages, Detroit — yielded only small
impacts. It thus appears that a minimum level of enforcement
by program staff is required to produce at least moderate
earnings and welfare impacts, presumably because this extra
“push” is needed in order to engage in program activities
those who normally would not participate on their own initiative.
While many programs achieved positive effects
on employment, earnings, and reduced use of welfare, few achieved
large effects, except for Portland. The Portland program
was unusually successful in substantially increasing employment
and earnings, helping people to get “good” jobs, lowering
welfare receipt, and achieving these outcomes for a cross
section of sample members. The results are probably due to
a combination of factors. While its employment message was
strong, the program offered high-quality education and training
services as well as job search, enforced a participation mandate,
and had strong job development and placement services. In
addition, contextual factors may have contributed to the program’s
success. In particular, it worked with a less disadvantaged
welfare caseload (relative to the other studied programs)
and operated within a good labor market with a relatively
high state minimum wage.
The
remainder of this summary details these findings. First, however,
it describes the key welfare-to-work program approaches contrasted
in the analysis and explains the evaluation’s research design
and samples.
II. Program Approaches
and Implementation Features
As
noted above, the evaluation’s sites implemented very different
programs; in fact, the research designs in several of the
sites were set up to rigorously compare the effects of specific
program approaches. This section discusses the two key implementation
features used in this report to define four broad program
approaches. In addition, for context, other major program
dimensions are described.
A. Employment-
or Education-Focused
Since
the late 1960s welfare-to-work programs seeking to increase
welfare recipients’ self-sufficiency have emphasized one of
two strategies. One strategy emphasizes quick employment,
reflecting the belief that individuals can best build their
employability, and eventually achieve self-sufficiency, through
actual work, even if their initial jobs are minimum wage and
without fringe benefits. The other strategy emphasizes skill-building,
particularly in the education area, reflecting the view that
individuals should first invest in education or training to
enable them to eventually obtain higher-wage, longer-lasting
jobs with health insurance coverage. The 11 NEWWS programs
blend elements of both strategies to varying degrees.
As
shown in Exhibit ES-1 four programs (Atlanta, Grand Rapids,
and Riverside LFA and Portland) were “employment focused.”
They provided job search assistance to a large segment of
their caseload and encouraged enrollees to find work as quickly
as possible. Further, both the Portland and Riverside programs
employed full-time job developers to help place program enrollees
in unsubsidized jobs.
The
three LFA programs, however, differed from Portland’s program
in important ways. The LFA programs routinely assigned individuals
to job search assistance, usually job club, as their first
activity, whereas Portland’s program offered GED preparation
classes to those deamed by case managers to have a good chance
of attaining a GED certificate relatively quickly. (Activities
initially assigned are an important clue to the “treatment”
experienced by welfare recipients, as many people leave welfare
or become exempt or temporarily excused from welfare-to-work
programs prior to being assigned to a second program activity.)
Further, Portland case managers encouraged enrollees to hold
out for jobs that paid well above the minimum wage and offered
the best chance for long-lasting and stable employment. In
contrast, case managers in the LFA programs, especially in
Riverside, stressed the value of starting off with any job,
even a low-paying one, and then advancing toward more stable
and better-paying jobs in the future.
Seven
programs (Atlanta, Grand Rapids, and Riverside HCD; Columbus
Integrated and Traditional; and Detroit and Oklahoma City)
can be characterized as “education-focused.” (See Exhibit
ES-1.) A large percentage of enrollees in these programs were
initially assigned to some type of skill-building activity.
The types of activities to which enrollees were first assigned
depended, in part, on the level of educational attainment
that individuals had achieved prior to entering the program.
Those who had not completed high school or received a GED
certificate but who were assessed by case managers as having
high school-level skills were assigned to GED preparation
classes. Those with lower reading or math levels were assigned
to Adult Basic Education classes. In addition, non-English
speakers could be assigned to English as a Second Language
(ESL) programs. Finally, those who had completed high school
or held a GED certificate could be assigned to vocational
training or employment-oriented skills courses at local community
colleges. All in all, however, assignments to GED preparation
or basic education courses were more common than assignment
to vocational training programs in these education-focused
programs, primarily as a result of welfare recipients’ low
levels of educational achievement; enrollment in college played
an even smaller role.
Some
differences existed among the seven education-focused programs.
The three HCD programs usually assigned enrollees to education
or training programs as their first activity. Case managers
in Columbus, Detroit, and Oklahoma had more discretion over
activity assignments, but, in practice, most program enrollees
were initially assigned to education or training activities
in these sites as well. Riverside’s HCD program was also unique
among this group in that it did not serve high school graduates
and GED holders who, at program entry, scored above minimum
levels in reading and math tests.
B. High or Low Enforcement of the Participation
Mandate
The
degree to which a program enforces a participation mandate
can be viewed as a product of three factors: how wide a cross
section of the welfare caseload is enrolled in a program;
how closely a program monitors individuals’ participation;
and how swiftly and consistently a program imposes financial
sanctions, that is, reductions in monthly welfare grants,
on those who do not participate.
All
four employment-focused programs, and five of the seven education-focused
programs, can be considered high enforcement programs; the
remaining two education-focused programs, Detroit and Oklahoma
City, can be considered low enforcement programs. While technically
requiring enrollment from a cross section of their “mandatory”
caseloads, these latter two programs put a priority on working
with those individuals who expressed interest in participating
in the program. In addition, resource constraints kept staff
in these sites from closely monitoring individuals’ participation
in program activities. Finally, staff in these two sites rarely
invoked financial sanctions. In contrast, program staff in
the other programs generally enrolled and worked with a cross
section of the welfare applicants and recipients who were
required to participate; monitored participation more closely;
and, especially in Columbus and Grand Rapids, frequently invoked
sanctions for nonparticipation.
C. Other Key Program Features
Other
implementation features, beyond those discussed above, can
also potentially influence a program’s effectiveness. Two
of them — the level of child care support provided and the
structure of program case management — are described here.
All
11 studied programs offered child care assistance to welfare
recipients who needed it while they were participating in
program activities or employed. Oklahoma City, Portland, and
Detroit provided the strongest staff support for arranging
child care. Staff in these programs helped to make child care
arrangements and also helped those who found jobs to obtain
transitional child care assistance. In contrast, case managers
for both Riverside programs did not provide much assistance
in setting up child care arrangements, encouraged enrollees
to use low- or zero-cost informal child care while they were
participating in program activities, and did not actively
promote the use of transitional child care benefits.
The
programs also differed in their case management strategies.
Columbus Integrated, Portland, and Oklahoma City implemented
an “integrated case management” staffing arrangement. That
is, case managers in these sites combined responsibilities
normally performed by income maintenance staff (determining
welfare eligibility, calculating welfare grants, invoking
financial penalties, and arranging for transitional benefits)
with responsibilities usually assigned to welfare-to-work
program staff (assigning enrollees to employment-related activities,
arranging for child care, and monitoring participation). Columbus
Integrated and Portland staff had sufficient resources and
small enough caseloads to perform both of these roles, enabling
them to promote a consistent self-sufficiency message. In
contrast, in Oklahoma City limited resources and large caseloads
led case managers to put most of their overall emphasis on
the financial functions of their job.
The
Atlanta, Grand Rapids, and Riverside LFA and HCD, Columbus
Traditional, and Detroit programs all used a traditional case
management structure, in which each welfare recipient had
two different case managers. Commonly, income maintenance
workers knew little about the welfare-to-work program in their
site. Among these sites, the staffing division was most pronounced
in Detroit.
III. Research Designs and Samples
In
Atlanta, Grand Rapids, and Riverside welfare recipients were
randomly assigned to either an LFA or an HCD program group
or to a control group. (See Exhibit ES-2.)
Both types of programs operated simultaneously in these three
sites. In Columbus a three-group random assignment design
was used as well. Here, the two program groups represented
two case management models: integrated and traditional. The
remaining three sites in the evaluation — Oklahoma City, Detroit,
and Portland — used random assignment to test the effectiveness
of established programs, as opposed to programs designed to
meet research protocols; individuals were randomly placed
in either a group that entered the program or a nonprogram
control group. Note that control group members were eligible
for child care assistance similar to that offered to program
group members if they were participating in nonprogram activities
in which they had enrolled on their own.
Individuals
were randomly assigned to research groups over approximately
a two-year period in each site. Random assignment for the
evaluation began in June 1991 in Riverside and ended in December
1994 in Portland. Thus, the results presented in this report
cover the calendar period from June 1991 (the first sample
member’s entry into the study) through December 1996 (the
last month of the two-year follow-up for the last sample member
randomly assigned in Portland).
Differences
in research design and random assignment procedures affected
the composition, and thus comparability, of the samples across
sites. (See Exhibit ES-2.) In five
of the seven sites AFDC applicants and recipients were randomly
assigned while attending a program orientation; in the other
two sites (Columbus and Oklahoma City) individuals were randomly
assigned before they were referred to a program. Since some
individuals typically exit welfare for employment or other
reasons before attending a program orientation, the samples
in Columbus and Oklahoma City include a larger share of individuals
who quickly left welfare.2
The
programs also differed in how broadly or narrowly they targeted
enrollment. Most notably, Oklahoma City randomly assigned
only welfare applicants (that is, persons in the process of
applying for welfare), including those whose application for
assistance was not yet approved. Additionally, Detroit, Grand
Rapids, Oklahoma City, and Portland extended their program
coverage to mothers with children as young as age 1, whereas
the remaining programs exempted parents whose youngest child
was under age 3. Riverside limited enrollment in its HCD program
to individuals determined by program regulations to need basic
education because they lacked a high school diploma or GED
certificate, attained low scores on a reading or math exam
administered at program entry, or had limited proficiency
in English. Finally, other pro-grams limited enrollment (and
thus those eligible for random assignment) by capping caseloads
for program staff and establishing waiting lists for enrollees
(Atlanta) or by excluding those who, in the judgment of program
staff, had serious barriers to participation (Portland).
Because
of these and other factors, the research samples differed
across the seven sites in key background characteristics likely
to affect individuals’ chances of finding employment and leaving
welfare. For instance, excluding the Riverside HCD program,
the proportion of sample members who had completed high school
or attained a GED certificate prior to random assignment ranged
from 55 percent (Oklahoma City) to 66 percent (Portland);
the proportion who had ever worked full time for at least
six months for the same employer ranged from 43 percent (Columbus)
to 77 percent (Portland); and, excluding Oklahoma City, between
28 and 50 percent of sample members in the sites had received
welfare cumulatively for five years or more.
IV. Findings
A. Program Participation and Enforcement
· Many control group
members took part in education and training activities on
their own initiative. All programs, however, were able to
increase participation levels in employment-related activities
above the control groups’ rate of activity during the two-year
follow-up. The size of the increase was associated with the
degree of enforcement of the participation mandate, but not
with the program approach.
Between
19 and 42 percent of control group members surveyed in each
site reported participating during the two-year follow-up
period in an employment-related activity, such as basic education,
skills training, post-secondary education, or formal job search.
As shown in Exhibit ES-3, all programs
increased participation beyond these levels of self-initiated
activity, from 9 to 40 percentage points above control group
participation levels. Overall, program participants were generally
involved in activities for at least several months.
All
but one of the programs with high enforcement of the participation
mandate (including both employment- and education-focused
programs) produced large impacts on participation (above 20
percentage points). Participation impacts were much smaller
for the two low enforcement programs (Detroit and Oklahoma
City). In these two sites the programs’ efforts increased
the number of welfare recipients who participated in activities
only slightly beyond what they would have done on their own,
in the absence of a mandatory welfare-to-work program.
·
As expected, all of the employment-focused programs produced
large increases in participation in job search activities.
Some also produced small increases in participation in education
and training.
The four employment-focused programs increased job
search participation by 27 (Grand Rapids LFA) to 32 percentage
points (Portland and Riverside LFA), compared with control
group levels. (See Exhibit ES-4.)
The programs achieved large gains for people who entered the
program with a high school diploma or GED certificate and
for nongraduates. Enrollees in the employment-focused programs
could be assigned to short-term education or training if they
completed job search without finding employment (or, in Portland,
at program entry). The Atlanta LFA and Portland programs produce
small increases in participation in education or training.
·
Most of the education-focused programs raised participation
levels in education or training. To a lesser extent, the programs
also increased participation in job search.
As
shown in Exhibit ES-4, the education-focused
programs increased participation in education or training
by 10 to 35 percentage points (Oklahoma and Riverside HCD,
respectively) compared with control group levels. (Detroit’s
increase in education or training participation was not statistically
significant.)
While
the increases for some programs were small when all sample
members are considered, most of the education-focused programs
achieved large increases in participation in education or
training for sample members lacking a high school diploma
or GED certificate at random assignment (not shown in Exhibit
ES-4). Most of these increases are accounted for by participation
in basic education.
When
enrollees in the education-focused programs completed education
or training, they were often assigned to job search. As Exhibit
ES-4 illustrates, all of the education-focused programs
raised job search participation levels to some extent; impacts
were similar for high school graduates and nongraduates.
·
Most programs produced only small increases in participation
in work experience or on-the-job training.
TANF participation requirements encourage states
to enroll welfare recipients in unpaid work or on-the-job
training. None of the programs in the evaluation made extensive
use of these activities, but most were able to produce small
impacts on participation in such activities because even fewer
control group members participated in them. (Participation
impacts on these activities are not shown in Exhibit
ES-4.)
·
The 11 programs varied widely in their use of financial
sanctions, or AFDC grant reductions, to enforce mandatory
participation requirements. Sanction rates in most of the
employment-focused programs were moderate, but rates in the
education-focused programs ranged from very low to very high.
In
three of the four employment-focused programs (Atlanta and
Riverside LFA and Portland) between 11 and 18 percent of program
group members reported that they were sanctioned for noncompliance
with program participation requirements during the follow-up
period. Three education-focused programs (Grand Rapids HCD
and Columbus Integrated and Traditional) and an employment-focused
program (Grand Rapids LFA) had high sanction rates, ranging
from 26 to 32 percent of program group members. At the other
extreme, almost no program group member in the low enforcement
education-focused programs in Detroit and Oklahoma City reported
being sanctioned.
B. Receipt
of Education or Training Credentials
·
Some of the education-focused programs, as well as the
Portland program, produced relatively large impacts on GED
certificate attainment among sample members who entered the
program without a high school diploma or GED certificate.
As
noted above, most education-focused programs increased participation
in basic education among nongraduates, but only three of these
programs (Grand Rapids and Riverside HCD and Columbus Traditional)
increased GED certificate attainment for this subgroup. Impacts
on GED receipt ranged from 8 to 11 percentage points. Notably,
Portland achieved similar gains in GED receipt. (The other
three employment-focused programs had no effect on GED attainment.)
·
For those entering with a high school diploma or GED,
a few programs increased the proportion who received a trade
license or certificate. One program increased the proportion
of nongraduates who received a trade credential.
Two
education-focused programs (Atlanta and Grand Rapids HCD)
and one employment-focused program (Atlanta LFA) increased
receipt of a trade license or certificate for sample members
in the graduate subgroup. Impacts ranged from 5 percentage
points (Atlanta LFA) to 11 percentage points (Atlanta HCD).
Portland increased receipt of a trade license or certificate
by 12 percentage points among those entering the program without
a high school diploma or GED. (Only Portland’s program
had this effect for the nongraduate subgroup.)
C. Employment
and Earnings
·
Employment-focused programs produced larger gains in
employment over the two-year follow-up period than most of
the education-focused programs.
Six
of the seven sites in the evaluation experienced economic
growth and strong labor markets during the first years of
follow-up; aided by these conditions, a majority of control
group members in these sites (from 58 to 72 percent) worked
for pay at some point during the two-year follow-up period.
Jobs were much harder to find in Riverside; only 45 percent
of control group members were employed during the follow-up
period.
As
shown in Exhibit ES-5, all four employment-focused
programs increased two-year employment levels, from 5 percentage
points (Atlanta LFA) to 15 percentage points (Riverside LFA).
(Exhibit ES-5 shows outcomes for both
program and control groups and the differences between the
two groups’ outcomes, that is, the impacts; other exhibits
present only the impacts for the various outcomes discussed.)
As described above, education-focused programs delayed job
finding in the short term. Not surprisingly, employment gains
for most of these programs fell below those of the employment-focused
programs. Three of the seven education-focused pro-grams produced
no statistically significant increase in employment (Columbus
Integrated and Traditional and Oklahoma), and the other education-focused
programs increased employment between 3 and 9 percentage points
(Atlanta and Riverside HCD, respectively).
·
Employment-focused programs produced much larger gains
in earnings over the two-year follow-up period than education-focused
programs.
Earnings
for control group members in the seven sites averaged between
$3,133 and $6,892 (including zeroes for those with no earnings)
over the two-year follow-up period. As Exhibit
ES-6 illustrates, Portland increased earnings by an average
of $1,842 per program group member. This earnings gain is
much larger than that of the other three employment-focused
programs and exceeds that of all previously evaluated mandatory
welfare-to-work initiatives, except the Riverside GAIN program
of the late 1980s (another employment-focused, varied first
activity program). Earnings gains in the other employment-focused
programs in the evaluation were moderate, ranging from $813
to $1,276 (Atlanta LFA and Riverside LFA, respectively). Earnings
gains in the education-focused programs were smaller; statistically
significant gains ranged from $367 to $677 (earnings impacts
in Riverside HCD and Oklahoma were not statistically significant).
Neither of the two low enforcement programs (Oklahoma City
and Detroit) produced substantial earnings increases.
·
Over time, the employment and earnings gains diminished
in most of the employment-focused programs, but increased
in most of the education-focused programs. By the end of the
two-year follow-up period some of the education-focused programs
had “caught up” to the employment-focused programs.
The
earnings gains in two of the three LFA programs (Grand Rapids
and Riverside) diminished over time, as increasing numbers
of control group members began finding jobs on their own.
In the last quarter of year 2 the three LFA programs raised
employment levels by only 4 percentage points and increased
average earnings by about $100. (Exhibit
ES-7 shows employment levels over the follow-up period,
averaged across programs within each approach.)
In
contrast, gains increased in most of the education-focused
programs. By the last quarter of year 2, impacts on employment
and earnings for five education-focused programs (Atlanta
and Grand Rapids HCD, Columbus Integrated and Traditional,
and Detroit) were similar to or slightly larger than impacts
for the three LFA programs: employment gains ranged from 3
to 6 percentage points and earnings gains ranged from $93
to $179. Overall, these results underscore the importance
of tracking the effects of education-focused programs over
a longer period than two years.
Unlike
the effects in other employment-focused programs, in Portland
positive effects on employment and earnings increased over
time: in the last quarter of follow-up the program group employment
level was 11 percentage points higher than the control group
level, and the program group earned on average $310 more.
These impacts are far larger than those of any other program
in the evaluation.
·
Portland’s program produced the largest, most consistent
increases in employment stability and job quality during the
follow-up period.
Portland’s
employment-focused, varied first activity program increased
the proportion of people who worked all four quarters of year
2 by 8 percentage points and who earned $10,000 or more in
year 2 by 6 percentage points. At the end of year 2 (as measured
from survey responses) the program increased the percentage
of people working at full-time jobs and at jobs that offered
health coverage. It also increased average hourly pay for
those working, but this finding, since it is based on a nonexperimental
comparison (different types of individuals in the program
and control groups may have been working) is more speculative.
The Riverside LFA program also increased full-time employment
with health benefits and higher hourly earnings, but to a
lesser extent than the Portland program. Contrary to expectations,
the education-focused programs increased job quality to only
a small extent or not at all by the end of two years.
D. Public Assistance Receipt and Payments
·
All programs reduced AFDC receipt to some degree. On
average, decreases for the employment-focused programs were
larger, but decreases for some education-focused programs
rivaled or exceeded decreases for some employment-focused
programs.
All
programs lowered the proportion of welfare recipients who
would have reached a two-year welfare time limit, had one
been in effect. Control group members in all but one site
received AFDC for an average of 16 to 20 months during the
two-year follow-up period. (The exception was Oklahoma City,
where the all-applicant sample averaged 12 months of receipt.)
The programs reduced the average number of months of AFDC
receipt by 0.48 to 2.41
months (or 2 to 16 percent).
Two employment-focused programs, Grand Rapids LFA and Portland,
produced the largest decreases (2.21 months and 2.41 months,
respectively). Decreases for the education-focused programs
ranged from 0.48 to 1.58 months.
In
the last quarter of follow-up between 41 percent (in Oklahoma
City) and 74 percent (in Detroit) of control group members
received an AFDC check. Portland produced the largest reduction
in the proportion of sample members receiving AFDC at this
point (12 percentage points). Among the other programs, reductions
in the proportion receiving AFDC at the end of year 2 ranged
from 6 to 7 percentage points for the three LFA programs and
from 3 to 7 percentage points for the education-focused programs.
·
All programs but one decreased average AFDC payments
over the two-year follow-up period.
Control
group members received AFDC payments over the two years averaging
between $3,624 and $10,302 (including those who left welfare
during the two-year follow-up period). Three employment-focused
programs (Grand Rapids and Riverside LFA and Portland) and
one education-focused program (Riverside HCD) reduced payments
by more than $1,000 (representing decreases of 10 to 19 percent,
relative to payments to the control group). (See Exhibit
ES-8.) Three other programs, all education-focused (Grand
Rapids HCD and Columbus Integrated and Traditional), also
reduced two-year welfare expenditures per program group member
by 10 percent or more. Detroit’s program produced only a slight,
not statistically significant, decrease in AFDC payments over
the two years.
·
Most programs reduced Food Stamp receipt and expenditures
during the follow-up period.
Eight
of the 11 programs decreased average Food Stamp expenditures
over the two-year follow-up period and decreased the proportion
of people who received Food Stamps in the last quarter of
year 2. Decreases in two-year expenditures ranged from 2 to
13 percent and decreases in receipt at the end of follow-up
ranged from 4 to 8 percentage points. One employment-focused
program and two education-focused programs had no effect on
Food Stamp receipt (Atlanta LFA and HCD and Oklahoma City).
E. Employment and Welfare Status at the
End of Two Years
·
In all programs a substantial proportion of enrollees
were receiving AFDC at the end of the two-year follow-up period.
Across all programs as many as 7 in 10 program
group members (in Detroit) remained on welfare at the two-year
mark. Even in programs that moved the largest proportion of
sample members off welfare, at least 4 in 10 enrollees remained
on welfare. This offers a caution to states striving to achieve
very rapid self-sufficiency for virtually all welfare recipients.
·
Most programs increased the proportion of people who
were working and not receiving AFDC at the end of the follow-up
period.
In
the last quarter of year 2 between 13 and 27 percent of control
group members were employed and receiving no AFDC payments.
All programs but two (Riverside HCD and Oklahoma City) increased
the proportion of people in this status. (See Exhibit
ES-9.) Impacts were generally small, with two programs
(Portland and Columbus Integrated) achieving moderate increases.
The impacts, which ranged from 2 to 9 percentage points, were
not associated with program approach.
·
Several programs representing both approaches slightly
increased the rate at which individuals left welfare without
a job.
The
proportion of control group members who were not employed
and not receiving AFDC in the last quarter of year 2 ranged
from 12 to 37 percent. All of the employment-focused programs
and three of the seven education-focused programs increased
the percentage of sample members in this status at the end
of two years. (See Exhibit ES-9.)
Increases were small in every program, ranging from 2 to 5
percentage points. The majority of people in this status reported
having some other source of income and/or living with someone
else who worked or who had another source of income.
F. Income and
Poverty
·
Most programs had little or no effect on income.
In the second year of follow-up
control group members averaged between $5,596 (Oklahoma City)
and $9,322 (Detroit) in combined income from earnings, AFDC,
and Food Stamps. Few programs substantially altered these
combined income levels; in general, reductions in AFDC, Food
Stamps, and other benefits matched or exceeded earnings gains.
However, in three programs — Grand Rapids and Riverside LFA
(employment-focused) and Riverside HCD (education-focused)
— combined income in the second year of follow-up was reduced
by $230 to $571, or 3 to 7 percent. (See Exhibit
ES-10.) In Portland (employment-focused) and Atlanta HCD
(education-focused) combined income increased in the second
year by $425 and $295, or 5 and 4 percent, respectively. This
combined income measure includes estimates of the EITC; when
EITC estimates are not included, losses and gains are somewhat
smaller, the Portland and Atlanta HCD gains are no longer
statistically significant, and a small loss in the Grand Rapids
LFA program becomes statistically significant.
·
Because income changes were minor, few programs lifted
many families out of poverty. Some programs, however, had
the effect of pushing a small proportion of families deeper
into poverty.
By
design, the combined income from welfare and Food Stamp grants
provides less than poverty-level income. Only by working can
people hope to attain enough income to escape poverty. In
the second year of follow-up between 11 and 26 percent of
control group members had combined income from earnings, AFDC,
Food Stamps, and estimated EITC receipt that equaled or exceeded
the federal poverty level. Five programs increased the proportion
of people living at or above poverty by a small amount. (See
Exhibit ES-10.) Portland was the
most successful, producing a 7.5 percentage point gain; impacts
for other programs were small, ranging from 2 to 3 percentage
points. Program-control differences for most of the other
seven programs were positive but very small and not statistically
significant.
In
the second year of follow-up between 19 and 48 percent of
control group members had combined income, including estimated
EITC, totaling less than 50 percent of the poverty line. Six
programs (both employment- and education-focused) slightly
increased the proportion of sample members living below 50
percent of the poverty level; they led to increases of between
2 and 6 percentage points in the proportion of individuals
living deeply in poverty.
G. Health
Care Coverage and Child Care Expenses
·
Some programs that increased employment levels and decreased
welfare receipt also decreased reported rates of health care
coverage.
At random assignment, almost all
sample members in the evaluation had health coverage because
they were receiving AFDC and were automatically covered under
Medicaid. (In Oklahoma City, applicants for assistance whose
eligibility was not yet determined were included in the sample,
so initial coverage rates there were lower.) Over time, coverage
rates declined for both program and control group members,
as some people left AFDC and did not replace their Medicaid
coverage with coverage from employers or other sources. By
the end of the follow-up period between 81 percent (Columbus)
and 88 percent (Detroit) of control group members reported
having health coverage for themselves and their children.
(This range covers all sites except Oklahoma, where 68 percent
reported coverage for themselves and their children.)
Two employment-focused programs
(Riverside LFA and Portland) and one education-focused program
(Columbus Integrated) that increased employment and decreased
welfare receipt at the end of follow-up period also lowered
health care coverage levels by 4 to 7 percentage points. (Impacts
in Portland were not statistically significant, but were just
beyond the .1 level of statistical significance used as the
standard throughout this report.) Although many program group
members who left AFDC (and automatic Medicaid coverage) found
jobs that provided health insurance, received Transitional
Medicaid benefits, or obtained alternative sources of coverage,
others were not able to replace the coverage they had under
Medicaid. Some of these respondents never received Transitional
Medicaid, and others had exhausted or had not restarted their
benefits at the end of the two-year follow-up period.
Program group members in Oklahoma
City reported even larger decreases in coverage — 11 percentage
points. This program decreased welfare receipt and appears
to have increased short-term employment — in jobs not reported
to the states’ unemployment insurance system — that did not
provide health insurance, especially for sample members’ children.
The other seven programs in the evaluation did not affect
health coverage rates for respondents or children.
·
Some programs increased child care use while employed
and out-of-pocket child care expenditures, an increase due
to greater child care use among those who found jobs as well
as an overall increase in employment levels.
Between
29 and 44 percent of all control group members (including
those who never worked) used child care while employed at
some point during the two-year follow-up period. Seven programs
— the four employment-focused programs and three of the seven
education-focused programs — produced moderate to large increases
in child care use while employed, ranging from 4 to 13 percentage
points. Impacts on paid child care use, that is, care paid
for by either the sample member, the welfare department, the
father of the child(ren), or the sample member’s employer,
were found in nine programs and were similar in magnitude.
The
increases in child care use and in paid care use while employed
are not entirely explained by the programs’ impacts on employment;
in many programs, of those who worked during the follow-up
period, a greater proportion of program group members than
control group members used child care (or paid care) as well.
The likely explanation for this finding is that employed program
group members required or preferred more stable child care
arrangements than employed control group members. This could
be partly due to differences in the characteristics of the
jobs acquired by program and control group members (for example,
program group members’ jobs were more likely to be full time).
It is also possible that program group members heeded the
messages they were given by their caseworkers — messages probably
delivered more frequently to program than control group members
— concerning the importance of obtaining paid, stable child
care.
Relatively few
program and control group members used transitional child
care benefits. Five programs increased the use of such benefits,
but these effects were large only in Atlanta LFA and Portland,
where increases of 7 and 11 percentage points in the receipt
of these benefits, respectively, were found.
H. Well-Being
of Children
·
Some of the welfare-to-work programs affected children,
although the effects were not large or consistent across outcome
measures or programs. Notably, the found effects on children
were both positive and negative.
The
NEWWS Evaluation is one of the first random assignment evaluations
of mandatory welfare-to-work programs to examine programs’
effects on the well-being of children. The children of sample
members in the evaluation were often quite young. As noted
earlier, in three of the sites women with children as young
as age 3 were required to participate in welfare-to-work programs;
in the other four sites the mandate was extended to include
women with children as young as age 1. Because many of the
child outcome measures used in the evaluation pertained only
to children of school age, however, the child impacts discussed
here are primarily for the subgroup of sample members who
had no children under age 6.
Control
group members in the seven sites had, on average, two to three
children. Across the sites an average of one-quarter of the
control group members in the subgroup with no children under
age 6 reported that at least one of their children had been
suspended from school at some point during the two-year follow-up
period. A smaller share of control group members — 8 percent
to 23 percent, depending on the site — reported having a child
who had repeated a grade in school during the follow-up period.
A relatively small proportion of all control group members
— less than 8 percent in any site — reported that a child
had been removed from their care during the two-year follow-up
period.
On
measures of children’s behavioral adjustment, such as suspension
from school, eight of the programs produced at least one statistically
significant effect on children among the subgroup of families
with no children under age 6. | 
