Publications

Issue Focus

Building Evidence on Employment Strategies for Low-Income Families

05/2018

Testing Programs Designed to Increase Employment, Earnings, and Economic Security 

Many Americans struggle in the labor market even when overall economic conditions are good. Unemployment is persistently high for some demographic groups and in certain geographic areas, and a large proportion of working-age adults – about 1 in 5 in 2017 – tend to be out of the labor force. In addition, in recent decades broad economic trends have dramatically reduced the availability of goodpaying, stable jobs for workers with low levels of education. Even people who work steadily often have difficulty making ends meet.   

In this context, in 2017, the Office of Planning, Research, and Evaluation within the Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services launched the Building Evidence on Employment Strategies for Low-Income Families (BEES) project to evaluate the effectiveness of innovative programs designed to boost employment and earnings among low-income Americans. Intended to build on previous research on the effectiveness of various employment strategies, the BEES project has the potential to fund up to 21 rigorous evaluations of promising programs serving recipients of the Temporary Assistance for Needy Families (TANF) program or other similarly low-income families who are not receiving TANF cash assistance. In addition, BEES will prioritize evaluations of programs that are state-initiated and programs that serve adults whose employment prospects have been affected by opioid dependency, abuse of other substances, or mental health conditions. The goal of the project is to strengthen ACF’s understanding of evidence-supported programs that are effective in improving employment and economic security. The project is being conducted by MDRC, a nonprofit, nonpartisan research organization, along with Abt Associates and MEF Associates.  

What kinds of programs is BEES studying? 

The BEES project has a broad focus on programs that aim to improve employment outcomes for lowincome adults. Programs could be operated by states, counties, nonprofit organizations, educational institutions, or others. BEES is designed to evaluate programs that are relatively mature, serve a substantial number of people, have stable funding, and have some evidence of success, such as a previous evaluation or strong outcome data. 

How will programs be evaluated? 

All of the BEES evaluations will include an implementation analysis and an impact analysis. The implementation analysis will document how the program operates on the ground, how managers and staff address operational challenges, and how much service participants receive. The impact analysis will assess the extent to which the program improves participants’ outcomes – for example, their employment rates and earnings. To measure program impacts, the evaluation team plans to use a random assignment research design, when possible, since it is the most rigorous method appropriate  for evaluating programs; administrative records and surveys will be used to track participants’ outcomes.    
 

Information for Administrators  and Program Operators 

The evaluation team is scanning the nation for promising programs that might be candidates for evaluation. This document provides some key information for programs that may be interested.  

How will programs benefit from participating in BEES

All programs that participate in the Building Evidence on Employment Strategies for Low-Income Families (BEES) project will get a state-of-the art impact and implementation evaluation, as well as funding to help defray any research-related costs. They will be part of a national project that will generate important evidence about employment strategies for low-income families, and they will have opportunities to interact with and learn from other programs in the project. Programs can use the evaluation findings to strengthen and improve their program models, and in an environment where public and private funding is increasingly being channeled to programs with evidence of success, positive results could lead to additional resources for expansion or replication. Programs will receive technical assistance from the evaluation team on issues such as recruitment and engagement, and they will also have access to a web-based Management Information System that can be adapted to meet program and research needs.  
 
How will programs be identified and selected for BEES? 

The evaluation team is conducting a national search for promising programs that might be good candidates for BEES. States, funders, program operators and other stakeholders are encouraged to nominate programs for the project. The evaluation team will conduct conference calls and, potentially, site visits to promising programs. Programs that are a good fit will be recommended to ACF for inclusion in the evaluation and, if selected, program managers and the evaluation team will jointly develop a plan to conduct the evaluation.   How will the BEES impact evaluation work? 

The evaluation team has extensive experience conducting random assignment studies in operating programs, and is sensitive to the needs of program managers, staff, and clients. The research procedures will be tailored to the specific circumstances of each program and designed to minimize the disruption of program operations. The evaluation team will be responsible for collecting follow-up data on study participants using administrative records and surveys. Participating programs will be asked to host site visits by the evaluation team, provide data on individuals’ participation in program activities and, where appropriate, help the evaluation team access existing administrative records. The evaluation team will provide feedback to participating programs throughout the course of the evaluation. In some cases, the evaluation may contribute to a continuous improvement process; for example, a program may be tested, tweaked to address its limitations, and then tested again.  

Want to learn more? 

ACF: Tiffany McCormack ([email protected]) or Carli Wulff ([email protected]) Evaluation team: Dan Bloom ([email protected]), Karin Martinson ([email protected]), Mike Fishman ([email protected])  

OPRE  MDRCABTMEF Associates