This is the fifth and final report from a multi-year evaluation
of Ohio’s Learning, Earning, and Parenting (LEAP) Program.(1)
Developed and operated by the Ohio Department of Human Services
(ODHS), LEAP is a statewide initiative that employs financial
incentives in an attempt to increase school enrollment and
attendance among pregnant teenagers and custodial teen parents
on welfare (almost all of them are women). LEAP, which began
operating in 1989, requires these teens to stay in school
and attend regularly or, if they have dropped out, to return
to school or enter a program to prepare for the General Educational
Development (GED), or high school equivalency, test. The program
thereby strives to increase the proportion of teens who graduate
from high school or receive a GED, find jobs, and ultimately
achieve self-sufficiency. These longer-term goals are important
because, even though teen parents make up fewer than 10 percent
of all Aid to Families with Dependent Children (AFDC) case
heads, families started by women who first gave birth as teenagers
account for approximately 50 percent of all long-term AFDC
recipients.
During the period of
this study (the rules have recently been modified), teens
who met LEAP’s requirements had their welfare checks
increased — $62 for school enrollment and an additional
$62 each month they attended school regularly (2)
— and teens who did not (without an acceptable reason)
had $62 deducted from their welfare grant every month until
they complied with program rules. Those who exceeded the allowed
number of total absences in a month but not the allowed number
of unexcused absences qualified for neither a bonus nor a
"sanction" (as such grant reductions are called).
Teens could be temporarily exempted from LEAP’s requirements
for medical reasons, to care for an infant, or if child care
or transportation was unavailable. Teens were no longer subject
to LEAP’s requirements when they reached the age of 20,
left AFDC, or received a high school diploma or a GED. During
1992 — approximately the midpoint in the period covered
by this report — a teen living on her own with one child
(the most common situation) was eligible for a monthly AFDC
grant of $274. Thus, a bonus raised her grant to $336 and
a sanction reduced it to $212. If she went from being sanctioned
to receiving a bonus, she would experience a 58 percent increase
in her welfare grant.
Teens’ enrollment
and attendance are monitored by case managers, who explain
the program’s rules, offer guidance, and authorize assistance
with child care and transportation teens may need to attend
school. LEAP itself provides no other services, although many
Ohio high schools have special programs, called GRADS (Graduation,
Reality, and Dual-Role Skills), which are designed to assist
teen parents in managing their dual roles as parents and students.
Also, as part of the evaluation, an enhanced version of LEAP
was put in place on a pilot basis in some Cleveland high schools;
it offered school-based services such as child care and intensive
case management as well as special GED classes and other services
for teens who were not complying with LEAP.
The LEAP evaluation
began with the start of the program itself and encompassed
12 Ohio counties, in which the large majority of LEAP teens
resided. Previous reports described the program’s operations
and analyzed LEAP’s effectiveness in achieving its education
goals and in moving teens toward employment and self-sufficiency.
The primary purpose of this final report is to present longer-term
(four-year) effects of LEAP on the teens’ employment,
earnings, and AFDC receipt and to assess the program’s
cost-effectiveness.(3)
The report is based
on a study of 4,151 teens who were identified as eligible
for LEAP, in all 12 research counties, during the program’s
second year of operation (August 1990 through September 1991).
As part of the study, these teens were randomly assigned,
by chance, to one of two groups: a program group, which
was subject to the LEAP program, or a control group,
which was not. Data were collected on both groups during a
four-year follow-up period. For employment and welfare outcomes,
this follow-up period started in the first month of the calendar
quarter in which the teen was randomly assigned. For outcomes
measured with the two follow-up surveys, the follow-up period
started in the month of random assignment itself. This means,
for example, that four-year employment outcomes for a teen
randomly assigned in December 1990 would cover the period
from October 1990 through September 1994. On the other hand,
three-year data on school completion would cover December
1990 through November 1993.
Because teens were
assigned to the two groups through a random process, there
were no systematic differences between the groups’ members
when they entered the study; any differences that emerged
between the two groups during the follow-up period can reliably
be attributed to the LEAP program. These differences are referred
to in this report as the program’s effects, or "impacts."
In September 1996,
based on its operational experience and informed by the flow
of findings from the evaluation, ODHS implemented changes
in LEAP to build upon and improve the program’s achievements.
For example, with the goal of increasing graduation rates,
an additional $62 bonus for completion of each grade (except
12th grade) and a graduation bonus of $200 have been added;
with the goal of changing the behavior of hard-to-reach teens,
those who are sanctioned for six consecutive months now have
their needs and their children’s needs deducted from
the calculation of the welfare cash benefit each month until
they comply with LEAP’s rules. (If they are on another
person’s case, that person’s portion of the cash
benefit is unaffected.) Although this report does not include
a study of these recent changes, it does discuss the results
in light of them.
The larger national
context has changed as well, with enactment of the Personal
Responsibility and Work Opportunity Reconciliation Act of
1996 — the landmark welfare law that replaced the entitlement
to AFDC with block grants to states. Under the law, unmarried
custodial teen parents under the age of 18 who have not graduated
from high school or received a GED or comparable credential
may not receive federally funded welfare assistance unless
they attend high school or a program that prepares them to
earn an alternative education or training credential. Prior
to the law’s enactment, some 25 states had already instituted
some form of school requirement for teen parents on welfare.
As one of the first programs implementing a school attendance
mandate for the state’s entire teen parent welfare caseload,
and as the first to be evaluated with a rigorous, random assignment
design, LEAP served as a model for many of these subsequent
efforts. The 1996 welfare law is likely to intensify interest
in the program and in the evaluation results as those who
design and develop new state programs and policies seek lessons
applicable to their own goals and circumstances.
The LEAP evaluation
was conducted by the Manpower Demonstration Research Corporation
(MDRC) under contract with the Ohio Department of Human Services.
Additional funding for the study was provided by the Ford
Foundation, the Cleveland Foundation, BP America, the Treu-Mart
Fund, the George Gund Foundation, the Procter & Gamble
Fund, and the U.S. Department of Health and Human Services.
The Findings in Brief
Previous reports found
that LEAP was successfully implemented, itself a genuine achievement.
The sheer scope of LEAP — a statewide initiative intended
to reach the entire target population — was virtually
unprecedented, and the existing management information system
(called CRIS) was not designed to permit the ready identification
and tracking of eligible teens or the administration of the
bonus and sanction system itself. LEAP also required entirely
new linkages between the county welfare departments and the
education system, which provided the enrollment and attendance
data needed to implement the bonuses and sanctions. Over time,
the program achieved smooth operations, with the phasing in
from 1991 to 1992 of a highly sophisticated statewide public
assistance computer system (CRIS–Enhanced). But because
the research sample entered the program before LEAP was fully
implemented, the reported results may be a conservative
estimate of the program’s potential.
LEAP’s immediate
goals were to induce dropouts to return to high school or
to enroll in GED programs, and to induce those still enrolled
to remain in high school. By linking bonuses and sanctions
to school attendance, the program also sought to promote better
attendance among those enrolled. To a considerable degree,
LEAP met these goals: It significantly increased school enrollment
and attendance, the outcomes directly linked to the incentives
provided, for both in-school teens and dropouts. During a
three-year follow-up period, the program also achieved positive
impacts on school progress (completion of the 9th, 10th, and
11th grades).
LEAP’s longer-term
goals included higher rates of high school graduation and
GED receipt, which it was hoped would increase teens’
employment and reduce their receipt of welfare. Here, the
results were more mixed. In general, the impacts on school
completion and employment were more positive for teens who
were in school when the program identified them as eligible
(the "intially enrolled" teens) than for those who
had dropped out before LEAP reached them (the "not initially
enrolled" teens). Specifically, the program did not increase
high school graduation rates except for the initially enrolled
group in Cleveland. LEAP did have positive effects on GED
receipt, but only for the initially enrolled group.
In spite of its limited
school completion impacts, LEAP was successful in increasing
the employment rates of initially enrolled teens throughout
the four-year follow-up period, and had positive impacts on
their earnings during the first two years, after which the
control group caught up. No employment impacts were found
for the not initially enrolled teens. Both groups experienced
significant reductions in welfare receipt, which were comparable
to welfare savings in welfare-to-work programs for adults.
LEAP’s impacts on earnings and welfare receipt also were
calculated separately for each county. In general, there was
no statistically significant variation in the impacts across
the various counties.
All
in all, the LEAP program benefited initially enrolled teens
by increasing their school attendance, GED receipt, and work
experience, whereas impacts for the not initially enrolled
were limited to school enrollment and attendance. LEAP achieved
these results while at the same time being cost-neutral for
the welfare department.
Table
1 highlights some of the key impacts, which are discussed
further in the remainder of this summary.(4)
The Data Sources and Sample for This
Report
Part of this final
report reviews and summarizes analyses presented in previous
reports. These analyses used several data sources: (1) a survey
of 1,188 teens, administered approximately one year after
each of them had entered the study (i.e., had been randomly
assigned to the program group or control group), and focused
primarily on the teens’ early experiences in LEAP and
their school enrollment status at the one-year point; (2)
a review of 263 LEAP case files from Cuyahoga, Franklin, and
Hamilton counties (Cleveland, Columbus, and Cincinnati) to
study LEAP’s implementation in general and the bonus/sanction
process in particular; and (3) a three-year survey of 913
teens in seven counties three years after random assignment
to measure education outcomes (including school attendance,
graduation rates, and GED receipt) and employment and welfare
outcomes. New analyses presented in this report focus on longer-term
(four-year) employment and earnings outcomes and AFDC receipt,
and are based on administrative data from the statewide Unemployment
Insurance and welfare systems, respectively. For the first
time, data for the impact analysis were available for all
12 research counties in which teens were randomly assigned
to the LEAP program. The 12 counties are Cuyahoga, Franklin,
Hamilton, Jefferson, Lawrence, Lorain, Lucas, Montgomery,
Muskingum, Stark, Summit, and Trumbull. They were selected
through a weighted random assignment procedure that assured
the inclusion of the three counties with the largest estimated
LEAP caseloads — Cuyahoga, Franklin, and Hamilton. About
two-thirds of the statewide teen population targeted by LEAP
resided in these 12 counties.
The sample for these
new analyses included 4,151 teens (3,479 program group members
and 672 control group members), randomly assigned between
mid-August 1990 and September 1991, i.e., those who entered
the study in the second (and last) year of random assignment.
This sample was chosen for two reasons. First, teens who entered
the study later experienced a more mature (though still evolving)
LEAP program than those who entered in the program’s
first year of operation. Second, for administrative reasons,
AFDC and employment and earnings data were not available for
the early part of the follow-up period: The employment data
cover the last three-and-a-half years of the four-year period;
the AFDC data cover years 3 and 4 only. By limiting the analysis
to those randomly assigned after mid-August 1990, it was possible
to present a consistent impact analysis, covering the same
length of follow-up for all sample members. The three-year
survey sample — the primary data source for the previous
report — was also drawn from teens who were randomly
assigned in the second year of random assignment, and, therefore,
is a subsample of the research sample for the present report.
Teens’ Initial School Enrollment
Status as a Context for the Findings
Many of this report’s
analyses are broken down by school enrollment status at random
assignment. Findings are presented separately for teens who
were enrolled in high school or in a GED program when they
entered the study (usually labeled "initially enrolled"
in this report, and constituting about 55 percent of the sample)
and for teens who were not enrolled in school or in a GED
program (labeled "not initially enrolled" or also
referred to as "dropouts"). Even though, after random
assignment, many teens moved from one status to the other
(i.e., either re-entered school or a GED program when they
were dropouts, or dropped out after having been initially
enrolled), the previous report found that initial school status
was an important predictor of program success.
In previous reports,
it was hypothesized that the incentives provided through the
LEAP program may have been sufficient to keep more of the
enrolled teens in school, but may not have been strong enough
to induce more of the teens who had left school to return
and remain enrolled. Doing so could have represented a major
(and, possibly, costly) change in the lives of the teens,
perhaps especially for those who had been out of school a
long time. Such teens may not have been willing to make this
change in return for the monetary incentives provided. Also,
the fact that some teens were not initially enrolled, while
others were, may reflect underlying obstacles to school enrollment,
which the teens would have had to overcome in order to return
to school. For example, the not initially enrolled teens tended
to be older, were less likely to live with a parent, and had
more children to care for.
Findings on Program Implementation
- The state and
county welfare departments successfully implemented LEAP
statewide, forging a link between welfare departments and
schools and putting the incentive system in place.
As discussed in detail
in the 1993 report and summarized in this one, LEAP’s
incentive structure was successfully implemented in each of
the 12 research counties. Program operations improved over
time, which meant that most teens in the research sample were
exposed to a more efficient and predictable LEAP program in
the latter part of the follow-up period. The key was full
implementation of a statewide computer system that made identifying
and tracking teens easier and carrying out bonuses and sanctions
largely automatic. Since the end of the follow-up period,
counties have also continued to improve teens’ access
to child care and other support services.
- Almost all eligible
teens (93 percent) were touched by LEAP’s incentives,
with 75 percent earning at least one bonus and 56 percent
qualifying for at least one sanction.
As discussed in previous
reports, a review of a sample of LEAP case files in three
counties found that fully 93 percent of these teens earned
at least one bonus or sanction, with the average teen qualifying
for about six grant adjustments (3.5 bonus payments and 2.8
sanctions) during her first 18 months in LEAP. During this
18-month period, there were more bonuses than sanctions: 37
percent of teens earned only bonuses; 18 percent qualified
for only sanctions; and 38 percent earned at least one bonus
and one sanction. In other words, 75 percent of teens earned
at least one bonus and 56 percent qualified for at least one
sanction. As time passed and the teens got older, those who
were still eligible for LEAP received more sanctions than
bonuses, probably because teens who had graduated or received
a GED by month 18 (generally cooperative teens, who earned
frequent bonuses) were no longer subject to LEAP, leaving
a higher proportion of frequently sanctioned teens still subject
to the program.
- The majority
of teens with multiple sanctions reported diminished spending
on essentials for their families, especially clothing and
food. Most teens with multiple bonus payments reported spending
a large share of the additional money on their children.
Teens who were sanctioned
at least four times reported in the three-year LEAP survey
that the resulting welfare grant reductions had a material
effect on their families: 58 percent said that their families
had fewer essentials (most often clothing, food, and medicine)
because of the grant reductions. Moreover, the sanctions reportedly
affected the children at least as much as their teenage parents.
Teens replaced part of the income they lost to sanctions by
borrowing money (usually from their parents), applying for
other forms of public assistance (most frequently Food Stamps),
and seeking additional child support. Two-thirds of the teens
postponed paying bills, most often utilities bills or rent.
Among teens who received
at least four bonus payments, close to 90 percent reported
using the additional money on essentials, especially for their
children. Almost a quarter also reported being able to pay
for some "luxuries" such as new clothing and outings
(e.g., to the movies or to the zoo) for their children. These
teens also were better able to pay their bills and to save
some money, which they said was later used to obtain special
items for their children, buy household essentials, and cover
unexpected emergencies.
Findings on School Enrollment, Graduation,
and GED Receipt
- LEAP’s immediate
goals — and the behaviors directly targeted by its
financial incentives — were to improve the teens’
enrollment and attendance in school or in a GED program.
After one year of follow-up, LEAP achieved a substantial
increase in these outcomes, for both initially enrolled
and not initially enrolled teens. The program also produced
modest, but statistically significant, increases in completion
of the 9th, 10th, and 11th grades.
As shown in Table
1, LEAP increased the number of months the average teen
attended high school during the first year of follow-up, from
4.2 to 4.8 months — an increase of 14.3 percent relative
to the control group. There were increases for both the initially
enrolled teens (from 6.6 months for the control group to 7.3
months for the program group) and the not initially enrolled
(from 1.0 months to 1.5 months). The latter also increased
their enrollment and attendance in GED programs, from 0.9
months for the control group to 1.7 months for the program
group.
These impacts on school
enrollment and attendance were followed by modest increases
in completion of the 9th, 10th, and 11th grades.
- LEAP’s effect
on school completion — its longer-term education goal
— was found to be limited when measured three years
after teens entered the program. The program increased GED
receipt among teens who were enrolled in school when they
entered LEAP. There were no impacts on high school graduation
or GED receipt among teens who were not enrolled in school
when they entered LEAP.
LEAP’s subsequent
education outcomes were less promising. For the initially
enrolled teens, the program increased GED completion rates
(from 4.4 percent for the control group to 10.0 percent for
the program group), but did not increase high school graduation
rates, except in Cleveland (based on an analysis of school
records). There is some evidence that these
gains in Cleveland may have been partly attributable to the
enhanced LEAP services provided on a pilot basis in some of
its high schools (as described in detail in the 1994 report).
For the not initially
enrolled, the impacts on enrollment and attendance in GED
programs did not translate into a higher rate of GED receipt,
nor did the 11th-grade completion impact lead to an increased
rate of high school graduation, at least not within the period
covered by the three-year follow-up survey.
- Two-thirds of
the teens in the sample did not receive a high school diploma
or a GED certificate within a three-year follow-up period.
However, approximately one-sixth of the teens were still
in school at the end of that period and could have graduated
or received a GED subsequently.
A number of factors
probably explain the low rates of high school completion shown
in Table 1. Some
teens may not have completed all the courses required for
graduation, or may have opted to take a GED test instead of
further pursuing a high school diploma. The teens’ feelings
about school and their own future doubtless played a part.
In an earlier survey, a large proportion of LEAP teens reported
that their schools were unsafe, inflexible, and unsupportive.
Some also viewed their economic prospects as dim, with or
without a high school diploma or a GED. Other studies have
pointed to the situational and emotional problems that can
make school attendance difficult for teenage single mothers.
However, in light of
the teens’ youth, when school completion and enrollment
are considered together, significantly more LEAP teens than
control group members (51.6 percent compared with 46.5 percent)
had graduated from high school, received a GED, or were in
high school or a GED program at the time of the three-year
survey. By the end of the three-year follow-up period for
school outcomes, 30.4 percent of teens in the sample were
under age 20, and 17.5 percent of program group members, compared
with 14.5 percent of control group members, were in high school
or a GED program. Thus, LEAP’s impacts on high school
graduation and GED receipt may have increased somewhat after
the three-year follow-up period.
It is also important
to consider LEAP’s high school graduation impacts in
the context of the overall graduation rates in the same locales.
According to official data, the high school graduation rates
for all students in five large school districts (Cincinnati,
Cleveland, East Cleveland, Columbus, and Toledo) ranged between
27 and 45 percent in 1994. Lifting graduation rates among
LEAP teens to the prevailing levels in their schools would
be a noteworthy achievement.
Findings on Employment and Earnings
- Teens in both
the program group and the control group experienced substantial
growth in their employment rates and earnings during the
four-year follow-up period. By the end of follow-up, four
out of five teens had worked in a job covered by Unemployment
Insurance. Overall quarterly employment rates for such employment
increased from 17 percent in the third quarter of year 1
to 40 percent in the last quarter of year 4. If employment
not covered by Unemployment Insurance could have been taken
into account, the employment rates and earnings presented
in this report would have been higher.
In spite of their responsibilities
as parents, their youth, and their educational and economic
disadvantages, a large proportion of teens included in the
LEAP evaluation were employed (often full time) at various
points throughout the follow-up period. Quarterly employment
rates increased steadily for both LEAP teens and those in
the control group. Almost 40 percent of all the teens were
employed at some point during the last three months of follow-up
(as shown in Table 2)
and average quarterly earnings had grown to $568 (or $1,457
for those employed). Therefore, even though few sample members
could be considered self-sufficient by the end of the four-year
follow-up, many had begun to develop ties to the labor market,
and relatively few had not worked at all. These findings attest
to the motivation of many young parents in the sample to find
work and become part of the formal labor market.
- The LEAP program
increased employment for initially enrolled teens, but not
for teens who were not initially enrolled. These impacts
were strongest early in the follow-up period.
Table
2 summarizes the program’s impacts on employment.
The table shows that program group members who were initially
enrolled in school were more likely to be employed than their
control group counterparts during years 1, 2, and 4 of the
follow-up period. For the full sample, these impacts on employment
were small (and statistically significant only in year 2),
because they averaged positive impacts for the initially enrolled
group with small negative impacts for teens who were not initially
enrolled.
Employment impacts
for teens who were enrolled in school at the time of random
assignment were quite substantial, especially in years 1 and
2 of follow-up. In fact, these employment impacts are comparable
to those found for adult welfare recipients in welfare-to-work
programs evaluated by MDRC. These impacts are remarkable,
especially because they occurred in spite of these teens’
greater involvement in school during this time, suggesting
that, contrary to what one might expect, increased school
attendance may not reduce opportunities to work — at
least part time — for these teens.
- LEAP increased
initially enrolled teens’ earnings during the first
two years of follow-up, but the impacts became smaller in
later years.
Table
3 presents the program’s impacts on earnings (averaged
in are zero earnings for those teens who were not employed
at all). Teens who were enrolled in school at random assignment
experienced quite substantial earnings impacts through year
2 of follow-up. Earnings in the second half of year 1 were
$101 (41.9 percent) higher for initially enrolled teens in
the program group than for their counterparts in the control
group. In year 2, earnings impacts for this group were $228
(a gain of 28.0 percent). These earnings gains not only reflect
an increase in the number of initially enrolled teens who
were ever employed during these periods, but also indicate
higher earnings for those who were. Earnings impacts were
not sustained in later years because teens in the control
group caught up with teens in the program group and increased
their earnings as well. However, the overall four-year earnings
impact ($544) remained positive for the initially enrolled
teens, even though it was not quite statistically significant.
Findings on AFDC Receipt
- Rates of AFDC
receipt remained high throughout the four-year follow-up
period, but were declining over time. More than 60 percent
of all teens were on welfare for at least one of the last
three months of follow-up, and close to 30 percent received
AFDC continuously during the last two years.
In any given month
within the four-year follow-up period, a majority of teens
in the program and control groups were receiving welfare.
Quarterly rates of AFDC receipt declined from 100 percent
at random assignment to 79.1 percent after two years, reaching
60.9 percent in the last quarter of year 4, with program group
members being somewhat less likely to receive AFDC than control
group members in at least one month in most quarters. However,
at the end of the follow-up period, almost 22 percent of teens
(or one in three of those receiving welfare) combined welfare
with (some) work.
- During the final
two years of follow-up, LEAP reduced welfare receipt by
increasing the number of teens who were not receiving any
AFDC and reducing the number who were receiving AFDC in
every single month.
Figure
1 shows a distribution of the number of months of AFDC
receipt during years 3 and 4 of the follow-up period. The
black bars represent program group members, and the white
bars represent control group members. The figure shows that
LEAP increased the number of teens who left AFDC altogether
during this time (shown in the first pair of bars) and decreased
the number of teens who received AFDC in every one of the
24 months represented (shown in the last pair of bars).
- LEAP reduced
both the number of teens receiving AFDC and the amount of
AFDC they received. Reductions were somewhat smaller for
those not initially enrolled than they were for teens who
were enrolled in school when they entered LEAP.
Table
4 summarizes LEAP’s impacts on AFDC during years
3 and 4 of the follow-up period (data were unavailable for
earlier years). During this time, LEAP reduced the average
number of months of AFDC receipt by three-quarters of a month
(0.76 months), which amounts to a 4.7 percent reduction relative
to the control group average of 16.03 months. During the first
six months of the last year of follow-up, the reduction in
the rate of AFDC receipt peaked at 5.1 percentage points —
a reduction comparable to those achieved by welfare-to-work
programs for adult welfare recipients. This reduction in the
rate of AFDC receipt was also reflected in the amount of AFDC
received by teens in the LEAP program. The average amount
of AFDC received was reduced by $275, or 5.0 percent, during
years 3 and 4 of the follow-up period.
In contrast to the
employment findings, AFDC impacts did not vary a great deal
by school enrollment status at random assignment, although
impacts for teens who were not initially enrolled were somewhat
smaller and failed to reach statistical significance in some
cases. They also declined faster toward the end of the four-year
follow-up period.
- During the last
two years of follow-up, LEAP caused reductions in the amount
of AFDC received that exceeded the program’s positive
effects on earnings.
In years 3 and 4 of
follow-up, program group members’ cash income from AFDC
and earnings was $345 less than that available to teens in
the control group, whose cash income was $8,693 over those
two years. This 4.0 percent reduction (not quite statistically
significant) occurred because earnings gains among program
group members were not large enough to offset reductions in
AFDC payments. The loss of cash income from AFDC and earnings
was greatest for teens who were not initially enrolled ($586
on average) because they experienced welfare reductions without
any offsetting earnings gains. The corresponding loss for
initially enrolled teens was only $101, which was not statistically
significant. However, some of the reductions in combined cash
income from AFDC and earnings may have been offset by the
Earned Income Tax Credit (EITC), increases in the receipt
of Food Stamps, or other unmeasured sources of income.
The Costs and Benefits of LEAP
- The LEAP program
was relatively inexpensive, with an upfront investment by
the Ohio Department of Human Services of $1,388 per teen,
or $747 for 12 months. Costs were higher for initially enrolled
teens than for those not initially enrolled.
Over the four-year
follow-up period for this report, program group members were
in LEAP an average of 22.3 months, at a net cost to ODHS of
$1,388 per program group member, or $747 for 12 months. (Net
cost refers to the average cost per program group member minus
the average cost per control group member. Although control
group members were not eligible for LEAP, they were eligible
for its child care services.) The net costs were higher for
the initially enrolled group ($1,659) than for those who entered
the program as school dropouts ($1,067). This is because the
former were younger and thus in LEAP longer (24.0 months,
on average, compared with 19.9 for those not initially enrolled).
These costs translate into 12-month net costs of $830 (for
the initially enrolled) and $643 (for those not initially
enrolled).
Table
5 shows the net cost to ODHS per LEAP teen by component.
The case management/county administrative cost ($1,140 per
program group member), which accounted for most of the cost,
includes expenditures for all program-related activities on
behalf of LEAP teens: orientation, initial and yearly assessments,
referrals for child care, providing transportation stipends,
monitoring school enrollment and attendance, and arranging
for bonuses and sanctions. In addition, case managers frequently
offered guidance in their ongoing contact with teens. The
case management/county administrative cost also includes expenditures
for administrative supervision and general overhead.
LEAP’s incentives
themselves did not add to the program’s total net cost
because, on average, teens received slightly more sanctions
than bonuses, leaving only the administration of the incentives
as a cost to ODHS. The cost to ODHS of transportation for
LEAP teens to attend school was $134 per program group member,
and child care costs to ODHS averaged $258 per program
group member, slightly more than the control group average
of $185; 7.6 percent of program group members and 5.5
percent of control group members used child care paid for
by LEAP, which was limited to licensed providers and was available
only to teens attending school.
- Because of LEAP-induced
savings in AFDC, Food Stamps, and related Medicaid expenditures,
ODHS recovered its investment in the program over the four-year
follow-up period.
Benefits of the LEAP
program were estimated using the available administrative
earnings and welfare data, supplemented with approximations
of taxes paid, tax credits received, Food Stamps received,
and Medicaid expenditures incurred. As shown in Table
6, over four years ODHS recovered its investment in LEAP,
with a return of 99˘ per $1 invested. (In this context, net
gains and losses refer to what was spent and saved on program
group members compared with control group members.) The program
was somewhat more cost-effective for initially enrolled teens.
A related analysis
takes into account costs and financial benefits to taxpayers
as a result of the LEAP program. Again, these are "net"
costs and benefits — the average for the program group
minus the average for the control group — over the four-year
follow-up period. From this "taxpayer perspective,"
the costs were the ODHS costs plus $332 incurred by other
agencies (mainly $518 for the additional LEAP-generated attendance
in high school and in GED programs, which was partly offset
by savings in other employment-related services). Most of
the financial benefits were the savings in AFDC, Food Stamps,
and Medicaid. As shown in Table
6, taxpayers recovered 75˘ of every $1 invested in LEAP,
even after factoring in the additional education costs attributable
to LEAP’s getting more teens to remain in or return to
school.
- LEAP teens experienced
a net loss of $1,110 over the four years.
As suggested by the
earlier discussion of the program’s impacts on cash income
from AFDC and earnings, the story was different for LEAP teens,
who experienced a net loss of more than $1,100 during the
four-year follow-up period. As shown in Table
6, both initially enrolled and not initially enrolled
teens experienced losses (e.g., reductions in AFDC receipt,
Food Stamps, and Medicaid eligibility) that were not offset
by gains (e.g., from earnings and tax credits). Even though
the initially enrolled group experienced some positive earnings
impacts, reductions in AFDC and other public benefits were
also greater for this group. On the other hand, LEAP teens
may experience long-term benefits from the additional education
and education credentials they received. These long-term benefits
may outweigh the losses apparent in the four years covered
by the evaluation.
Implications for Policy and Program
Design
- The LEAP results
suggest that it is sensible to include a LEAP-like approach
as one element of a state’s welfare strategy.
LEAP has proved itself
to be a promising approach to increasing school involvement
among teen parents on welfare. As other states continue to
develop their teen parent strategies, they may want to pursue
similar approaches. The main challenge — one that ODHS
has addressed in its recent modifications of LEAP — is
not only to increase school attendance but, beyond that, to
improve high school graduation rates so that these young mothers
have a better chance of success in the labor market and a
route to self-sufficiency.
- The LEAP experience
highlights the importance of meeting the implementation
challenges inherent in such programs, particularly the need
for ongoing case management and well-designed management
information systems.
The LEAP program relies
on case managers who are dedicated to working with teen parents,
and who are supported by a computerized case information system.
Without ongoing case management, a program like LEAP would
be very difficult to implement: The attendance monitoring
role that case managers play is central to the program, as
is the role of providing assistance and information to teens.
In order to implement
programs like LEAP, it is also necessary to develop a well-designed
management information system. The need to integrate welfare
data, school data, and case management information to provide
correct and timely incentives places a substantial burden
on the information system available to program administrators.
With the 1996 federal welfare law requiring school attendance
by unmarried, custodial, minor teen parents receiving federal
Temporary Assistance for Needy Families (TANF) funds, other
states will be pushed to develop programs similar to LEAP,
and many will confront system development tasks similar to
those that faced ODHS.
- The importance
of initial school enrollment status as a predictor of program
success underscores the need to prevent teen parents from
dropping out of school. Recent program changes address this
problem by limiting LEAP’s pregnancy and age-of-child
exemptions, which could have accounted for some teens’
dropping out after becoming pregnant.
This report shows that
teens who were in school when they were reached by LEAP had
better outcomes and experienced stronger program effects than
teens who had dropped out of school before LEAP reached them.
Apparently, it is more difficult to work with teens after
they have dropped out of school already. Therefore, it is
important to develop interventions and policies that prevent
teens from dropping out in the first place.
One way that ODHS has
approached this issue is by changing the LEAP program’s
exemption policy. Previously, LEAP teens were exempt from
the program after the first trimester of pregnancy and up
to three months following the birth of their child. This means
that a teen could have been out of school for nine months,
which would have set her back academically and would have
increased the risk that she would drop out altogether. As
part of the program changes implemented in 1996, the pregnancy
exemption was eliminated (except in cases of medical need),
and the post-partum exemption was reduced to six weeks from
three months. It is hoped that these changes will reduce dropout
among LEAP teens.
- LEAP’s impact
on school attendance was encouraging, and would be strengthened
if more teens responded to the program’s incentive
structure. This outcome might be achieved by changing the
incentive structure and its implementation, and by directly
addressing teens’ reservations about going to school.
School attendance might
be further boosted by enhancing the teens’ general understanding
of the LEAP incentives, reinforcing their awareness of the
short-term financial benefits of meeting LEAP’s requirements
(partly by modifying procedures so that the bonuses can be
paid more rapidly), and continuing to stress the longer-term
benefits of completing one’s education. The size of the
incentive could also be increased. With sanctions and bonuses
combined, LEAP’s incentive was already substantial, but
ODHS’s decision to dramatically increase sanctions for
teens who do not comply with LEAP’s regulations for six
consecutive months is likely to increase compliance, albeit
simultaneously increasing the risk to teens (and their families)
who are unable or unwilling to meet LEAP’s requirements.
Some teens may not
stay in school (or re-enroll in school) despite LEAP’s
incentives because they feel that the cost of staying in school
or going back is simply too high to be offset by the incentives
offered to them. Some are working, and may experience a loss
of income if they return to school. Others do not want to
go back to school because they want to spend time with their
children or do not want to leave them in the care of strangers.
Yet others say that they consider school a dangerous place,
feel unwelcome there, or find it to be a poor environment
in which to learn. All of these practical considerations and
negative feelings represent a "cost" of returning
to school. Reducing these costs could further increase school
attendance. For example, teens could be allowed to receive
reimbursement for child care options they are comfortable
with (instead of relying entirely on licensed day care), and
the school environment could be improved.
Finally, it is important
to acknowledge that many teen parents have academic problems
or have dropped out before they became parents, which suggests
that their academic problems may not be related to their status
as teen parents. Other types of assistance, or more general
school reform measures, may be needed to help teens who drop
out for academic reasons prior to becoming teen parents.
- In addition to
stimulating school enrollment and attendance, programs like
LEAP should provide incentives that specifically reward
academic progress and school completion. Special school-based
services may be needed to improve school outcomes for teen
parents who respond to incentives and return to school.
Rewarded for "seat
time" with attendance bonuses, some of the teens who
responded to the LEAP program may not have had an incentive
to actually make academic progress, earn credits, and work
toward graduation. In fact, teens who regularly attended school,
and made steady academic progress, were in a sense penalized
for graduating or earning a GED because at that point they
could no longer receive bonus payments for continuing to attend.
This suggests that programs like LEAP should develop incentives
that emphasize school performance and graduation in addition
to seeking increases in school enrollment and attendance.
For example, ODHS has recently implemented grade completion
bonuses, which are paid when teens successfully complete one
grade and are promoted to another (except grade 12), and a
graduation bonus when teens successfully complete high school
or obtain a GED. Alternatively, bonuses could be based on
grades, which reflect both attendance and academic
performance.
LEAP’s high school
completion impacts may also be improved by providing tutoring
and remedial classes that help teens clear the hurdles associated
with earning a high school diploma, such as required courses
and school-wide achievement tests. Finally, incentives and
counseling should emphasize the importance of completing high
school, as opposed to dropping out to pursue a GED or to seek
employment instead.
- Special attention
is needed for teen parents who had dropped out but who returned
to school because of LEAP, and for older teens who were
far behind in their schooling when they dropped out.
The results of the
LEAP evaluation indicate that teens who were not enrolled
in school or in a GED program when they became eligible for
LEAP were at particularly high risk of making some academic
progress but then failing to graduate from high school. Special
attention is needed for these teens, including stronger efforts
by the schools themselves, to foster the retention and academic
progress of teen parents who are induced to return to school
with financial incentives and start out at a disadvantage.
Experience with LEAP
also made it clear that it often is neither feasible nor desirable
to place long-term dropouts who are not of compulsory school
age into low grade levels that fit their academic skills (e.g.,
placing an 18-year-old in the 7th grade). To address this
issue, ODHS’s recent changes in the program included
the provision that if no appropriate education activity can
be found for these teens, they will be transferred to the
JOBS welfare-to-work program and will be subject to JOBS participation
requirements.
- The 1996 federal
welfare law poses important challenges for policymakers
who develop and administer teen parent programs like LEAP.
In addition to getting teen parents to attend school, programs
must now address work requirements and time limits on receipt
of federally provided welfare assistance that will affect
young parents after they graduate or age out of programs
like LEAP. Therefore, these programs may have to combine
their education message and incentives with more employment-oriented
activities.
As states continue
to operate or launch learnfare programs for teen parents on
welfare under the new welfare rules, they will need to consider
the employment and welfare prospects facing these teens once
they leave school. As teens graduate or reach the age of 20,
they will likely become subject to the same work requirements
and time limits that may apply to adult recipients of welfare.
Therefore, it may be necessary for LEAP and other such programs
to implement services and incentives that encourage teens
to work, or make connections with the world of work, outside
regular school hours. Fortunately, the employment data presented
in this report show that many teens engaged in temporary and
part-time employment, even without an intervention that specifically
tried to make them do so. LEAP and similar programs could
build on these teens’ apparent eagerness to work by offering
opportunities for paid internships and part-time work during
the summer and after school. By integrating these opportunities
into LEAP and other programs, cooperation with those programs
could become attractive to more teen parents on welfare, thereby
enhancing the overall effectiveness of these programs.
- The problems
facing teen parents on welfare are substantial and complex.
LEAP successfully addressed one problem area by increasing
these teens’ school attendance. However, more needs
to be done to improve the self-sufficiency of these teens
and to reduce the persistent poverty among them. The LEAP
evaluation shows that there are no easy answers. More experimentation
is necessary, both within and outside the context of the
LEAP program.
By providing both positive
and negative financial incentives, the LEAP program uses an
innovative approach to induce teen parents to attend school.
This report showed that this approach holds promise. However,
by design it addresses only part of the problem facing teen
parents on welfare, and only some of the causes of their potential
long-term dependency. It will be important for governments
and program administrators to continue to experiment with
different approaches to changing the behavior and improving
the outcomes for teen parents and their children. While some
of these may be targeted specifically at teens and the problems
of teenage parenthood, others may have to address the broader
social and economic circumstances that affect the lives and
opportunities of low-income young people and the choices they
make.
By creating,
implementing, rigorously testing, and continuously seeking
to improve the LEAP program, the State of Ohio and the Ohio
Department of Human Services took a pioneering step. The program’s
designers and administrators well knew how complex and daunting
was the issue of teen parenting and the associated long-term
welfare receipt, but they also knew how little had been proven
to work in this area, and how critical was the need for promising
new approaches. ODHS has already put to work lessons from
this evaluation — about both the program’s strengths
and limitations — by making far-reaching changes to LEAP.
For Ohio and the nation, tracking how this modified
program works will be important to the development of responsible
and effective social welfare policy.
Notes
1.
The four previous reports are: Dan Bloom, Hilary Kopp, David
Long, and Denise Polit, LEAP: Im-plementing a Welfare Initiative
to Improve School Attendance Among Teenage Parents, 1991;
Dan Bloom, Ve-ronica Fellerath, David Long, and Robert G.
Wood, LEAP: Interim Findings on a Welfare Initiative to Improve
School Attendance Among Teenage Parents, 1993; David Long,
Robert G. Wood, and Hilary Kopp, LEAP: The Educational Effects
of LEAP and Enhanced Services in Cleveland, 1994; and David
Long, Judith M. Gueron, Robert C. Wood, Rebecca Fisher, and
Veronica Fellerath, LEAP: Three-Year Impacts of Ohio’s
Welfare Initiative to Improve School Attendance Among Teenage
Parents, 1996.
2. For
high school students, “regular attendance” was defined
as having no more than two unexcused absences and no more
than four total absences in a month.
3. Sample members
are referred to as “teens” in this report, but by
the end of the four-year follow-up period, most were in their
early twenties.
4. In this and
the other impact tables, statistical significance is indicated
by asterisks. A “statistically significant”
result is one that has less than a 10 percent probability
of having occurred simply by chance and not as a result of
the program.
|
