Design, Sites, and Data Sources
Family Rewards Pilot The Family Rewards pilot served 2,400 families, and an equal number of families were assigned to the control group. Eligible families lived in the selected community districts with incomes at or below 130 percent of the federal poverty level and had at least one child in public school in either the fourth, seventh, or ninth grade. These grades were chosen because they reflect critical transition points in students’ school careers. Students who fail to navigate them successfully fall behind their peers and have difficulty recovering, increasing their risk of dropping out, especially in the early high school years.
Seedco assembled a network of six Neighborhood Partner Organizations in the designated community districts to recruit and enroll eligible families. Although the program had no formal case management component, participating families had several avenues — a telephone hotline, a Web site, and an in-person help desk at the community-based organizations — to get information on services that could help them meet the program conditions (for example, information on where to get homework help for their children, how to find a doctor, and where to get help finding jobs and training). In addition, families could also get help opening bank accounts, so that the payments could be transferred electronically and accessed via debit cards.
The evaluation of ONYC: Family Rewards has three major strands of research: an impact analysis, an implementation and process analysis, and a benefit-cost analysis:
- Impact analysis. This analysis is examining the program’s effects on a wide range of outcomes, including children’s school performance; family health care practices and health outcomes; parents’ employment and training outcomes; and family income, benefit receipt, poverty, material hardship, and quality of life.
- Implementation and process analysis. This analysis is exploring the operations of Family Rewards, focusing particularly on the roles and experiences of the implementing institutions (particularly, Seedco and the community partners) and on the perceptions and experiences of the participating families.
- Benefit-cost analysis. This analysis will estimate the cost of operating Family Rewards, distinguishing how much was spent on various aspects of program delivery versus the amount of cash transferred to the participating families. It will also make a number of benefit-cost comparisons, examining the economic “gains” and “losses” from several perspectives, such as from the perspective of participants and their families and from the perspective of taxpayers and government budgets.
The first interim report on the program was published in March 2010. It presented initial findings from the implementation and impact studies covering the first two years of program operations. It showed that participating families received about $3,000 in rewards per year, with almost all families earning some rewards. The amount of money a family received varied according to the number of activities the family completed and compositional factors, such as family size and number of high school students in the family. The impact evaluation found a wide range of effects across a number of domains and outcome measures. There were strong effects in the areas of poverty reduction, school achievement for a subset of ninth-graders (who were somewhat more prepared for high school when they entered the study), and receipt of dental care. Other early effects in the areas of health and employment were modest or mixed, and the program has so far has not had an effect on school achievement for elementary and middle school students. The study team released a separate report in May 2011 that used in-depth qualitative interviews to explore families’ experiences with the program, including how families communicated about the educational incentives, and the influence of those incentives on parent-child interactions.
Work Rewards Pilot
A total of 3,987 Section 8 recipients were recruited for the Work Rewards study. Eligibility for the Work Rewards pilot was limited to voucher holders whose household income was no greater than 130 percent of the federal poverty line. However, eligibility was not limited to particular community districts but extended to voucher holders from across New York City. In addition, families with or without children under age 19 were invited to apply.
The program and evaluation design differed depending on the source of the participant’s Section 8 voucher. For holders of a New York City Housing Authority Section 8 voucher, the study will determine whether the offer of incentives for sustained full-time employment and for taking up approved education or training programs improves labor market and other outcomes. Applicants were randomly assigned to a program group that received the incentives offer and control group that did not receive the offer.
Holders of Section 8 vouchers through the City’s Department of Housing Preservation and Development were randomly assigned across three groups rather than two. One group was offered enrollment in an enhanced version of the Department’s Family Self-Sufficiency (FSS) program operated by a number of community-based organizations located throughout the city, with some additional services. A second group was offered enrollment in FSS and received the additional workforce incentives. The third group was a control group which was offered neither of these options.
The analysis comparing the FSS group versus the control group constitutes the first-ever random assignment test of the Family Self-Sufficiency program, the main federal program intended to increase employment and earnings and reduce reliance on government subsidies among Section 8 voucher holders. Comparing the first and second groups in this trial — those offered the FSS program versus those offered FSS plus the workforce incentives— will show whether the incentives “add value” to whatever effects FSS (including its escrow-based asset-building strategies) produces on its own.