This report presents an assessment of the implementation and the two-year impacts of a program in Texas that aimed to promote job placement, employment retention, and advancement among applicants and recipients to the Temporary Assistance for Needy Families (TANF) program. The program in Texas is part of the Employment Retention and Advancement (ERA) project, which is testing 15 programs across the country.1 The ERA project was conceived and funded by the Administration for Children and Families in the U.S. Department of Health and Human Services (HHS) and is also supported by the U.S. Department of Labor. The project is being conducted by MDRC, a nonprofit, nonpartisan research organization, under contract to HHS.
The ERA program in Texas was evaluated in three cities (sites) — Corpus Christi, Fort Worth, and Houston — beginning in the fall of 2000. Most of the employment outcomes presented in this report cover the first two years after individuals entered the program. The results include the program’s effects on employment levels and stability, earnings, and advancement in the labor market. These results are important but are not the final word on the program, as MDRC will track employment outcomes for the study’s participants for a total of three years.
The ERA Project
Although much is known about how to help welfare recipients find jobs, little is known about how to help them and other low-wage workers keep jobs or advance in the labor market. Previously studied postemployment programs were not found to improve participants’ outcomes. The ERA project was designed to build on past efforts and identify and test innovative programs designed to promote employment stability and wage progression among low-income groups, including welfare recipients. From 2000 to 2003, a total of 15 experiments were implemented in eight states, including Texas.
The design of the evaluation is similar in most of the project’s sites. Individuals who meet the ERA eligibility criteria, which vary by site, are assigned at random to a program group, called the ERA group, or to a control group. Members of the ERA group are recruited for (and, in some sites, are required to participate in) the ERA program, while those in the control group are not eligible for ERA services but can access other services and supports available in the community. MDRC is tracking both research groups over time. The random assignment process ensured that the two groups were comparable when they entered the study; thus, any differences between them that emerge over time — for example, in employment rates or average earnings — can be confidently attributed to the ERA program.2
The Texas ERA Program
The Texas ERA program was designed to promote job placement, retention, and career advancement for TANF applicants and recipients. The program provided both pre- and postemployment services and targeted a population applying for or receiving cash assistance, most of whom were not working when they entered the program. The Texas ERA program included job placement, employment stabilization, and advancement services — along with a monthly stipend of $200 for working TANF leavers, to encourage employment retention and advancement.
The Texas ERA program was developed by the Texas Department of Human Services (DHS), in coordination with the Texas Workforce Commission (TWC). DHS was primarily responsible for determining eligibility and overseeing the TANF cash assistance program, while TWC managed TANF employment services. TWC oversaw employment services for the ERA program as well as for Choices — the standard program in the state that provides employment-related services to TANF recipients. As designed, the Texas ERA program was to include the following services:
- Preemployment job search and team-based case management services. For most participants, the ERA program initially provided job search, job readiness, and case management services. Because postemployment case management services had little effect on employment outcomes in past retention and advancement studies, Texas strengthened these services by having them begin at the preemployment stage and using a team-based approach. The goal potentially was to involve partners from multiple agencies (including DHS, local workforce staff, and organizations working to prevent substance abuse or domestic violence) that had expertise in addressing specific employment-related barriers. Key case management services included employment assessment, goal setting and career planning, support services, resolution of employment barriers, and job search assistance.
- Stipend for welfare leavers. The ERA program provided a monthly stipend of $200 to participants who left TANF, were employed for a minimum of 30 hours per week, and participated in a postemployment “advancement” activity. The stipend was also available to those who combined 15 hours of work per week with an education and training activity that lasted 15 hours per week. Before receiving the stipend, participants had to first exhaust the four-month earned income disregard, which allowed all welfare recipients who left the rolls to continue to receive most of their full welfare grant for four months.3 There was a lifetime limit of 12 stipends (which did not have to be used in consecutive months). The stipend was included in the ERA model based on other studies that found similar earnings supplements to be effective in encouraging job retention and increasing earnings.4
- Intensive postemployment services. The ERA program provided a comprehensive set of postemployment services, which could include assistance with job-related problems and support services, monitoring job performance and issues through regular site visits to employers, rapid reemployment assistance for participants who lost jobs, and support in meeting the requirements of the stipend. ERA postemployment services could continue for as long as an individual was eligible for the stipend.
The Design of the Evaluation
Immediately following an eligibility or recertification interview for TANF (but, in the case of applicants, before they were approved for cash assistance), individuals were randomly assigned either to the ERA program or to the Choices program. Those who were assigned to ERA were introduced to the program and were then required to attend an orientation on the TANF program before being approved for cash assistance. Once these individuals were approved for TANF, they were then engaged in ERA program services. Individuals who were not approved for TANF were not eligible for the ERA program. Random assignment began in October 2000.
Once assigned to the ERA program, individuals completed an assessment; then they generally participated in job search and, if they did not find a job, community service (a volunteer position in a nonprofit or public organization); and then they received case management services. Once employed, participants received postemployment services, and — after leaving TANF and after receiving the earned income disregard — they became eligible for the monthly stipend. Participation in the ERA program was mandatory for most individuals, meaning that they faced a reduction or termination of TANF benefits for noncompliance with program rules.
Individuals in the control group were assigned to participate in Choices, the state’s standard welfare-to-work program, which also provided pre- and postemployment services to TANF recipients. In terms of preemployment services, individuals who were assigned to Choices — like their counterparts in ERA — also completed an assessment, participated in job search, and did community service if they did not find employment. Choices also provided case management and support services, but its staff generally did not use the team-based approach that was developed for ERA and did not typically engage in longer-term career planning. Again like ERA, participation in Choices was mandatory for most TANF recipients.
There were greater differences between ERA and Choices in the nature of postemployment services. Employed individuals in Choices were not eligible for the $200 stipend, although they could receive the earned disregard for up to four months (at the same level as under the ERA program). Under Choices, postemployment case management services generally lasted only for the duration of the earnings disregard (whereas services continued for up to an additional 12 months for workers receiving the stipend under ERA), and they were less intensive than the postemployment services provided through the ERA program.
Key Findings on Program Implementation
This section summarizes the report’s findings on how the Texas ERA program was implemented and on sample members’ participation in the program and other employment-related services. The findings are based on field research, a time study of ERA staff, automated program tracking data on stipend receipt, and a survey administered to a subset of sample members about 12 months after they entered the study. Key implementation findings follow.
- While ERA was intended to be distinct from Choices in terms of both pre- and postemployment services, most of the key differences were in the nature of the postemployment services, particularly the stipend.
In all the Texas sites, the job search and case management services, participation mandate, and support services provided under Choices resulted in the control group’s receiving a relatively strong set of preemployment services that were similar to those provided by ERA. The Choices program also experienced some improvements over the course of the study period. Although “team-based” case management (involving partners from multiple agencies) was an important concept in the development of the ERA program — and one that was intended to distinguish it from the Choices program — in the end the preemployment case management services provided by ERA were similar to those provided by Choices.
For those who found jobs, ERA’s retention and advancement services were strongest and most distinct from Choices after the four-month earnings disregard period — once individuals were receiving a stipend. At this point, participants began working most closely with the postemployment staff, typically meeting with them at least once a month. Program staff in Corpus Christi and Fort Worth often conducted this monthly meeting at the worksite of participants who were receiving a stipend — although employer site meetings did not start in Fort Worth until later in the study period.
- Overall, Corpus Christi achieved the highest stipend receipt rate, with 30 percent of the program group receiving at least one payment (compared with 20 percent in the other sites). All the sites in Texas found that significant effort had to be put into marketing the stipend.
Some individuals in the program group did not qualify for the ERA stipend because they did not find jobs or they did not work longer than the four-month earned income disregard period. Individuals who received the entire earned income disregard generally became eligible for the stipend — although they might not have received it in a given month if they did not work enough hours, submit the required documentation, or attend the required activity.
Figure ES.1 shows the steps involved in receiving a stipend in Corpus Christi. Out of every 100 people randomly assigned to ERA, 84 met the first criterion for receiving a stipend: They were determined eligible for the TANF program and received benefits. Of those 84 people, 70 became employed at some point during the follow-up period, but only an estimated 55 of them appear to have worked enough to receive the entire four-month earnings disregard.5 Of those 55 who worked at least four months, 30 people (55 percent) received a stipend. In Fort Worth and Houston, approximately 40 percent of those who worked longer than four months received a stipend (not shown in the figure).
All the sites in Texas developed a solid marketing effort for the ERA stipend, particularly over time, with Corpus Christi moving most quickly to develop marketing materials and strategies. However, with only about half of those who were eligible receiving the stipend, there was still clearly room for improvement. The reasons for not using the stipend among those who were eligible appear to have varied and included not attending the required employment activity, a desire to discontinue involvement with a government program, and a lack of knowledge or understanding about eligibility requirements. In addition, despite program services, it appears that job loss continued to be an issue, with some ERA participants losing jobs before they became eligible for the stipend. Notably, among those who did receive a stipend, many continued to receive it on an ongoing basis. In Corpus Christi and Fort Worth, almost half of those who received a stipend received it 11 or more times.
- Corpus Christi implemented the program most smoothly, and the other sites adopted some of its strategies over time.
Program practices in Corpus Christi included designating case managers to work only with individuals who were receiving stipends and developing strong postemployment services, including regular site visits to employers, in-house support groups to meet the stipend participation requirement, and specific performance measures for staff. These innovative practices reflect the very strong management team that the Corpus Christi program had in place during the early phases of the project. The Fort Worth program struggled for a good portion of the study period, but it made significant improvements when a new manager was hired, including implementing more structured job search services and making regular employer site visits. The Houston program moved the most slowly in getting key ERA components — particularly postemployment services — off the ground.
- Compared with Choices, the ERA programs in Corpus Christi and Fort Worth increased the proportion of ERA group members who received retention and advancement services.
The ERA retention and advancement services included help finding a better job while working, enrolling in life skills classes while working, and career assessment. Despite the increased participation in these services, the overall level of receipt is somewhat lower than expected: Only about 40 percent of survey respondents from the ERA program who were working reported that they had received such assistance. This could possibly reflect that — because some individuals did not receive the stipend and more intensive retention and advancement services until they had spent more than a year in the program — this activity was not fully captured by the ERA 12-Month Survey, which was used to measure service receipt.
In Corpus Christi and Fort Worth, the ERA program also resulted in significantly more contacts with case managers after participants entered the study, but, compared with Choices, the differences were not large. In general, participation rates in employment-related services were relatively high in all the Texas sites for both the ERA group and the Choices group.
Key Findings on Program Impacts
Table ES.1 summarizes the impacts of the Texas ERA program. Administrative records are used to examine whether ERA’s work incentive and pre- and postemployment services translated into improved employment rates, job retention and advancement, and total income — and into reductions in public assistance receipt. Unemployment insurance (UI) wage data and public assistance payment records are the primary sources for creating outcomes of employment, earnings, TANF, and food stamps and for estimating impacts on these outcomes. The main cohort for this report includes study participants who were randomly assigned from October 2000 through June 2002 (N = 4,288). This represents three-quarters of the eventual sample that will be analyzed in Texas. UI records are available for two years after program entry, and welfare and food stamp receipt are available for one and a half years. The report’s key impact findings follow.
- ERA did not produce consistent or large effects on employment or earnings outcomes. The ERA programs in Corpus Christi and Fort Worth, however, led to modest increases above the control group averages on some measures of employment and employment retention. The program in Houston had no effect on employment and earnings.
Outcomes for the control group represent what would have happened in the absence of the ERA program. In any given quarter, approximately half of control group members were employed. On average, they earned only $8,000 over the full two-year follow-up period. (This average includes zeroes for those who were not working.)
ERA did not increase average earnings over the two-year follow-up period. While ERA increased the percentage ever employed in Fort Worth, it did not increase measures of employment retention in any of the sites. This suggests that the ERA stipend did not increase employment and was paid mostly to those who would have worked anyway.
Yet the two-year impacts in Corpus Christi mask some modest impacts that were evident after one year. In Year 1, ERA increased average quarterly employment by over 3 percentage points and increased the proportion of ERA group members who were employed for four consecutive quarters — a key measure of employment retention — by over 4 percentage points (not shown). During Year 2, however, these effects were no longer statistically significant. Program impacts did not increase over time, as might have been expected.
The impacts also do not reflect variations in stipend receipt. In Corpus Christi, only about 16 percent of the ERA group received a stipend in Year 1 (not shown). In Year 2, approximately 24 percent of the ERA group received a stipend, but the impacts on employment retention measures were no longer significant.
In Fort Worth, ERA increased employment by 4 percentage points over the control group’s average of 76 percent over the two-year follow-up period. This impact was larger in Year 2, which may reflect the strengthening of the ERA program in that site, as noted above. In Year 2, ERA group members were 6 percentage points more likely to have been employed than control group members. However, there was no effect on measures of job retention.
For the most part, what impacts there were are not large or consistent. The weak impacts may be partly attributable to the fact that the ERA program was measured against a group of sample members who were engaged in a relatively strong welfare-to-work program.6 It should be noted that, by the end of Year 2, impacts on employment (in Corpus Christi) and on earnings (in Fort Worth) emerged (not shown). These impacts had not been evident in the preceding quarters, which makes it difficult to know whether they will persist into Year 3.7
- Due to the stipend, ERA increased total income in Corpus Christi but not in the other Texas sites. For the most part, ERA had no effect on receipt of TANF or food stamps.
Table ES.2 shows that the ERA program increased total income in Corpus Christi by $604 over the control group’s average of $11,247. Half of this increase is attributable to the stipend. Thus, it appears that the primary effect of the ERA stipend was to increase income (since the program did not encourage employment beyond what would have happened in the absence of the program).
- While the two-year impacts of the Texas ERA program have been weak, it is too early to be sure that the program has failed to achieve its goals.
In particular, because effects seemed to emerge at the end of Year 2 among the full samples in Corpus Christi and Fort Worth, and because the Year 3 impacts in Corpus Christi among an early cohort look positive, these results should not be considered the final word on the Texas ERA program. In addition, most stipend recipients began receiving the stipend in Year 2, and some were still receiving it in Year 3. There is a possibility that the impacts might improve later in the follow-up period. MDRC will continue to monitor the impacts of the ERA program in Texas.
Conclusions
The Texas ERA program is one of 15 being studied as part of the ERA project, and reports over the next two years will present results for other programs. MDRC will continue to track sample members in Texas and will make public longer-term results when they are available. As the study continues to generate information, more definitive conclusions will be possible. However, some preliminary conclusions can be drawn based on the results in this report.
- Employment retention and advancement among welfare recipients remains a goal that is difficult to achieve.
For the most part, the ERA program in Texas has been unsuccessful thus far in increasing employment retention and advancement — even in Corpus Christi, the city that most strongly implemented the program’s components. Impacts for certain subgroups and cohorts do suggest that, under certain circumstances, the program can have a modest effect on these outcomes. While it is too early to be sure, the results suggest that, in states with already-strong welfare-to-work programs, there are likely to be limited gains in “upgrading” to a program that follows the Texas ERA model.
- It is critical to consider the design and marketing of financial incentives.
Past studies have found that programs providing a financial incentive to encourage work among welfare recipients — such as the Minnesota Family Investment Program (MFIP) and the Canadian Self-Sufficiency Project (SSP) — have had large effects on increasing employment, earnings, and job stability and on reducing poverty.8 The weaker effects of the Texas ERA program may reflect several factors. First, qualifying for the financial incentive involved a series of steps, including working longer than four months, working enough hours, submitting the necessary documentation, and attending a monthly employment-related activity. Moreover, the more intensive postemployment case management (such as employer site visits) did not generally start until the stipend phase of the program — after an individual had been working for four months. These factors may have limited the number of individuals who became eligible for the stipend and may have resulted in the stipend’s going to those who would have begun to work and stayed employed without the stipend. While this is not necessarily a negative outcome (the stipend did increase income for these families), the main goal of the stipend was to increase employment stability. In contrast, the financial incentive in the MFIP program — which was provided as part of the welfare grant, as an earned income disregard — was automatically provided to all those who went to work, and it took effect immediately after the recipient became employed.
Second, while the Texas sites made a good-faith effort to market the ERA stipend, that may not have been sufficient. The financial incentive in the Canadian SSP program was also voluntary and required individuals to work 30 hours per week, but SSP had nearly universal participation among those who became eligible. In contrast to ERA, SSP had a more comprehensive marketing strategy, including a one-on-one orientation session dedicated exclusively to a discussion of the financial benefits of the incentive. Although marketing of the ERA stipend in Texas was a solid effort that grew stronger over time, information about the stipend was often provided along with a range of other information about program requirements and services. Marketing the ERA stipend may have been particularly challenging, given that receipt of the stipend could seem far in the future to some participants and was contingent on achieving several outcomes in addition to becoming employed.
Finally, it is important to consider that the magnitude of ERA’s impacts in Texas was likely affected by the control group program. Choices was a relatively strong work-focused welfare-to-work program, which may have produced a more difficult comparison group to “beat” than the control groups in MFIP and SSP.
Notes
1 Although the Texas program operated in various cities that are called “sites” in this report, Texas counts as a single ERA site.
2 For more information on the ERA project, see Bloom, Anderson, Wavelet, Gardiner, and Fishman, New Strategies to Promote Stable Employment and Career Progression: An Introduction to the Employment Retention and Advancement Project (U.S. Department of Health and Human Services, 2002). For early results from four sites, including Texas, see Bloom, Hendra, Martinson, and Scrivener, The Employment Retention and Advancement Project: Early Results from Four Sites (U.S. Department of Health and Human Services, 2005).
3 Depending on participants’ earnings, the amount of the residual TANF grant combined with the earned income disregard was roughly equivalent to or somewhat less than the $200 stipend. Individuals could receive the earned income disregard once in a 12-month period (but the four months did not have to be used consecutively within this period).
4 See Gordon Berlin, Encouraging Work, Reducing Poverty: The Impact of Work Incentive Programs (New York: MDRC, 2000).
5 Because individuals could receive the earned income disregard only once in a 12-month period (but the disregard did not have to be used in consecutive months), the population that was eligible for the stipend was estimated — using unemployment insurance (UI) data — as those employed individuals who worked in two or more quarters within a year and who earned over $2,400 in these two quarters (the equivalent of working at $7 per hour for 20 hours per week). This was done to eliminate those who worked very little over the two quarters and would not have received the disregard for the full period. Because the earnings threshold that is used to determine who would be eligible for a stipend is an estimate, a sensitivity analysis was conducted to determine how the results would change using different earnings thresholds. Overall, no large differences were found in the number who were eligible for the stipend.
6 Since the Texas ERA program began at the preemployment phase but was most different after sample members became employed, the impacts may have been slightly smaller than if the evaluation had been conducted only among sample members who were employed at the time of random assignment.
7 Analysis from an early cohort indicates that the impacts on employment may have strengthened in Year 3. Employment and earnings were examined for sample members, randomly assigned from October 2000 through June 2001, for whom an additional year of follow-up data from administrative records was available. In Corpus Christi, the three-year impacts on employment among this early cohort were statistically significant throughout the third year of follow-up. By the end of Year 3, impacts for this cohort approached 10 percentage points and were statistically significant at the 1 percent level. It is unclear why the impacts were stronger among this cohort. It is also unclear why the impacts strengthened later in the follow-up period. It should be noted that the sample sizes for this cohort are rather small (N = 668). Among the early cohorts in Fort Worth and Houston, there were no statistically significant effects in Year 3.
8 Berlin (2000).
|
