Parenting Behavior in a Sample of Young Mothers in Poverty: Results of the New Chance Observational Study

This is a report on the New Chance Observational Study – an in-depth examination of parenting behavior and its relationship to children’s development in a subset of the families participating in the New Chance Demonstration.

New Chance was a national research and demonstration project that provided comprehensive education, training, and other services intended to improve the prospects and well-being of low-income mothers and their children. The program’s eligibility criteria were designed to assure that the research sample represented families central to the welfare reform debates of the past 10 years: families headed by young mothers who had their first child as teenagers, were high school dropouts, and were receiving Aid to Families with Dependent Children. (AFDC was the main cash welfare program until the 1996 federal welfare legislation replaced it with Temporary Assistance for Needy Families, or TANF.)

One of New Chance’s distinguishing features was its explicit two-generational focus on both mothers and children. Many of its services were meant to help the mothers prepare for, get, and hold onto jobs so that they could become economically self-sufficient and leave welfare. In designing the demonstration, MDRC believed that such changes, if they occurred, would potentially improve developmental outcomes for children. But the aim was to shape the children’s development more directly. Therefore, the demonstration included parenting education, access to pediatric health services, and an attempt to develop and encourage the use of good-quality child care.

The original research plan for evaluating the New Chance Program recognized the importance of assessing its two-generational nature. Thus, it included several measures of parenting behavior and participation by the mother in the child-related components of New Chance (e.g., parenting education), along with various measures of the children’s development. Sources for the latter included surveys of parents, reports from teachers (for children in the research sample who were in formal child care or school settings), and a general measure of school readiness.

While the initial design of the New Chance Program and its research plan went beyond the practices typical of large-scale field studies of this type, they created the opportunity to push even further. Thus, MDRC formed an interdisciplinary "observational studies team" that designed and implemented the New Chance Observational Study and a companion study embedded within the evaluation of the Job Opportunities and Basic Skills Training (JOBS) Program (the welfare program authorized under the legislation that preceded the legislation establishing TANF).

Each of the observational studies makes a new and unique contribution regarding how best to measure parenting and child outcomes in a survey context. This is a question important to anyone trying to understand the effects on children of welfare reforms or similar initiatives. Much of the existing information about how children develop comes from the university-based laboratories of developmental psychologists, yet social policies and programs play out in the everyday lives of parents and children. The observational studies transported university-based techniques into the field, gathering data via videotape and audiotape in the participants’ homes. In large part, this translation of methods and materials succeeded, and it yielded rich information about the strengths and limits of using regular survey interviewers to go well beyond their typical interviewing tasks. The information on all this methodological work is found in the second part of this monograph.

The incorporation of these new techniques and measures into the New Chance Demonstration also allowed us to deepen our answers to certain questions. For example, using the originally planned survey measures, MDRC staff have examined questions about the effects of New Chance on parenting, and the relationships between those effects and child outcomes. But these analyses acknowledged the limits of data on parenting or child development that come from parental self-report and observations by survey interviewers. By including the measures derived from videotapes and audiotapes of the observational study sessions, the observational studies team was able to more fully explore the effects of New Chance on parenting, the relationships between program participation and these effects, and the role of the parenting effects on child outcomes.

Interestingly, the New Chance program did have positive effects on parenting. These appeared on both survey interviews and observational measures. However, other influences such as maternal psychological well-being, including stress and depression, combined with the effects of parenting behavior on children. Thus, the positive parenting differences were not sufficient to bring about effects on child outcomes, no matter how the parenting and child outcomes were measured.

While the observational study data were consistent with the survey data and thus did not change our judgments about the effects of the New Chance Program, they were promising in other ways. The observational data appear to have tapped a wider range of parent and child behaviors than is possible in the survey context. For example, the survey measures did not detect program impacts in the area of cognitive stimulation, while the more sensitive and qualitative observational measures did. The observational measures also added to our ability to understand why some children are doing better than others. Such questions and analyses are the subject of the first part of the monograph.

Interdisciplinary and pathbreaking, the observational study represented a collaboration among a diverse team. The editors of the monograph, Martha Zaslow and Carolyn Eldred, have generously described the roles of all team members except themselves in the Acknowledgments. Each of the editors has given an extraordinary amount of time to this effort over the last six years, and their contributions cannot go unnoted.

Martha Zaslow has been involved in all aspects of this work. Together with colleagues at Child Trends, she analyzed the interview-based measures of parenting, examined the relationships across different parenting measures, and conducted analyses regarding the role of parenting and other factors in predicting child outcomes. She also wrote or contributed to several chapters in the first part of the monograph and reviewed all of the manuscript several times.

Carolyn Eldred began the effort as the project director for the New Chance surveys at the Institute for Survey Research at Temple University. In that role, she worked with MDRC to hone the overall study design, thought through the myriad issues involved in conducting and recording the observational sessions, adapted the laboratory protocols for survey administration, trained the field staff, and directed the data collection effort. Subsequently, as a consultant to MDRC, she undertook the methodological reflection and analysis in the second part of the report, wrote the chapters on methodological issues, coauthored the chapter on study design in the first part of the report, and reviewed the entire monograph several times. Both editors have been "first among equals" on this team.

The study owes a great debt to the energy and good efforts of the mothers who invited us into their homes, diligently worked through the tasks with their children, and shared their thoughts in the interviews. Above all else, the research team hopes that this work is up to the trust shown by these families. They have been open and generous; this monograph is dedicated to improving their lives and the lives of others in similar circumstances.

Finally, we are grateful to the funders of the observational study: the Foundation for Child Development, the William T. Grant Foundation, the Smith Richardson Foundation, and the National Institute of Child Health and Human Development. Their commitment made the project possible.

The New Chance Observational Study — the subject of this monograph — is an in-depth examination of parenting behavior in 290 of the 2,322 families studied in the New Chance Demonstration, a national research and demonstration program operated between 1989 and 1992 at 16 locations in 10 states. The demonstration tested a program model intended to improve the economic prospects and overall well-being of low-income young mothers (aged 16 to 22) and their children through a comprehensive and intensive set of services. It was developed by the Manpower Demonstration Research Corporation (MDRC) and supported by a broad consortium of public and private funders.

New Chance was directed at families central to the welfare reform debates that culminated in the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 — families headed by young mothers who gave birth during their teenage years and were receiving Aid to Families with Dependent Children (AFDC, the main cash welfare program).¹ More specifically, New Chance focused on those who were especially disadvantaged because they were high school dropouts; as a group, they and their children are at high risk of long-term welfare receipt and economic hardship.

The New Chance Program sought to help the young mothers (who, for the most part, volunteered for the program) to acquire educational and vocational credentials and skills so that they could find and keep jobs offering opportunities for advancement and reduce, and eventually eliminate, their use of welfare. It also sought to motivate and assist participants to postpone additional childbearing and to become better parents. Because New Chance focused on young children as well as their mothers, it sought to further the cognitive, social, and emotional development as well as the health of participants’ children. Child care was provided at no cost to the parents, on site in most places, and the program facilitated access to health services for both mothers and children. The program was intended to be intensive (four to five days a week for up to 18 months), though in practice attendance was of much shorter term and often irregular.²

New Chance was one of only a few interventions for families in poverty that took a two-generational approach, seeking to improve the outcomes for both the young mothers, who faced multiple difficult life circumstances, and their children. For this reason, parenting behavior took on particular importance. If the New Chance Program improved the quality of the mother-child relationship, it would signify improved functioning in an important area of the mothers’ lives. It would also have the potential of diminishing risk and improving developmental outcomes for the children. Research shows that the children of young single mothers in poverty begin to show higher levels of behavior problems as early as the preschool years. Later in development, they show difficulties in school progress and achievement.

New Chance sought to enhance parenting behavior most directly through its parenting classes, which provided information on children’s developmental stages, activities and materials to enhance children’s cognitive development, and developmentally appropriate strategies for shaping child behavior. Parenting classes, which were scheduled for about two hours a week, balanced open discussion of issues of concern to the mothers and more formal presentation of specific information.

Other aspects of New Chance also held the potential for enhancing parenting behavior. Life skills training (like parenting classes, scheduled for about two hours a week) focused on improving the mothers’ skills in communicating with significant people in their lives, including children. Adult basic education (including preparation for the General Educational Development, or GED, test) and job skills training classes provided stimulation to the mothers, which in turn could be reflected in how they talked with, read to, and played with their children. Group and individual counseling addressed problems emerging in the mothers’ lives, including problems with children. Family planning classes stressed the importance of providing enough time and attention to each child in the family. The child care that the children participated in provided a context for the young mothers to observe care providers engaging in stimulating and supportive interactions with children — behavior they might then imitate. Participation in the program as a whole could increase the mothers’ sense of social support, which in turn could enhance parenting behavior.

Central to the New Chance Demonstration was a rigorous evaluation of the program’s effectiveness. For this purpose, 2,322 young women who applied to New Chance were randomly assigned to either an experimental group (who were allowed to enroll in New Chance) or a control group (who did not have access to services provided through New Chance, but many of whom found some alternative services in their communities). To determine the effectiveness of the New Chance "package" of services, compared with these alternative services, differences in outcomes between the two groups of women and their children (often referred to in this kind of research as the impacts of the program) were examined through structured survey interviews administered approximately 18 and 42 months after each young woman entered the research sample. The findings from the demonstration have been presented in a series of MDRC reports concluding with the final report, released in 1997.

Because of the importance of parenting behavior in the context of a program focusing on outcomes for two generations, a special study was undertaken, using videotaped observations of mother-child interaction among a subset of 290 families in the New Chance Demonstration. The work of a collaborative and interdisciplinary research team (described in the Foreword to this monograph), the observational study was funded by grants from the Foundation for Child Development, the William T. Grant Foundation, and the Smith Richardson Foundation, with specific analyses focusing on methodological issues funded by the National Institute of Child Health and Human Development.

The observational data were collected during a special session following the 18-month follow-up survey for the full evaluation, on average 21 months after each member of the observational study sample had enrolled in the demonstration. The videotapes of mother-child interaction were then coded under rigorous conditions in university laboratories. The purposes of the study were to (1) describe parenting behavior in this sample, especially the affective quality of mother-child interaction and the aspects of interaction that are related to the emergence of literacy in children; (2) examine in greater depth the program’s impacts on parenting behavior; (3) explore the role of parenting behavior in shaping the outcomes for children; and (4) assess the added value of using measures of parenting based on direct observation in addition to the evaluation’s survey interviews.

The study also asked how best to measure parenting behavior in a program evaluation that focuses on two generations. More precisely:

The sample for the New Chance Observational Study was chosen from the seven New Chance sites that had the largest number of families who met the study’s criteria and could potentially participate. Specifically, the study chose families with a "focal child" between 30 and 60 months old, an age range considered appropriate for the study’s procedures. (The "focal child" was the child in each family who was the focus of the interviews and assessments in the full evaluation’s impact research.) In addition, the observational study was limited to African American and white families, rather than also including Hispanic families, because the researchers did not consider it possible to give full consideration to variation in parenting behavior that might be associated with cultural background. To study how parenting behavior related to other important aspects of the families’ lives, the sample was also limited to families who had completed the evaluation’s 18-month follow-up interview. Finally, the study was restricted to families for whom the observational study session could be scheduled within a similar time frame relative to random assignment — no more than four months after each mother’s 18-month follow-up interview. The eligibility criteria for this study were more restrictive than the criteria for the overall New Chance Evaluation, and the sites used in the observational study were not selected randomly from all the New Chance sites. Therefore, the findings from the observation study should not be seen as generalizing to the full New Chance sample. The larger evaluation sample, for example, included families with children in a broader age range, families from additional sites, and families of Hispanic origin.

The sites from which the families were chosen were the Bronx, Detroit, Harlem, Lexington (Kentucky), Philadelphia, Pittsburgh, and Portland (Oregon). Within each of these sites, families who met the study’s eligibility criteria were contacted in the order in which they had enrolled in the demonstration.

The goal was to conduct observational sessions with about 300 families — a sample size large enough to detect program impacts on the observational measures and to permit an examination of how parenting behavior was related to the mothers’ education, age at first birth, years receiving public assistance, and other important characteristics, as well as child outcomes. Of the New Chance sample members who met the criteria for inclusion in the observational study, 79 percent participated, yielding usable videotapes from 290 families. Approximately 84 percent were African American and 16 percent were white. Among the focal children, 148 were boys and 142 were girls. One hundred and eighty-four of the mothers were members of the demonstration’s experimental group and 106 were from the control group. The families in the two groups did not differ significantly in their baseline characteristics, so group differences found at the time of the observational study can be attributed to the experiences of the families subsequent to random assignment. Nor was there evidence of systematic differences between the families identified as eligible to participate in the observational study who did and did not participate.

The visit to each family participating in the observational study (lasting, on average, about an hour) was conducted by a two-member team — a survey interviewer, who briefly interviewed the mother and guided her through a series of interactive tasks with her child, and a videographer, who taped the mother and focal child as they carried out the interactive tasks. The interviewers, who were accustomed to conducting traditional survey interviews and were already working on the New Chance 18-month survey, received specialized training to follow a structured script and to explain and administer the tasks to the mothers with minimal interference in the mother-child interaction.

The observational session, or extra visit carried out for the New Chance Observational Study, started with the interviewer explaining the procedures to the mother and obtaining her informed consent. The interviewer then administered the first half of a brief interview with the mother, in which she was asked to describe, in half-hour intervals, the activities that she and her child had engaged in during the previous weekday. Beginning the session in this way helped the interviewer to establish rapport with the mother, gave the videographer time to set up the equipment, and provided further information about the mother-child relationship.

Following a script, the interviewer then described each of the interactive tasks to the mother and confirmed that she understood how to carry out each task. After these initial instructions, the child was invited to join the mother and interviewer, and the mother was asked to guide her child through each of the interactive tasks. The script called for the interviewer to remind the mother briefly about each task as she presented the props for that task. Interviewers were instructed to then let the mother interact with her child as she chose, without interference.

The tasks, devised by university-based researchers of children’s development, had been used in previous studies of mother-child interaction, including studies of low-income families. The six tasks were:

Each of these tasks had either been used by members of the research team in previous studies or was a modification of a task used before. The tasks were chosen because they yield forms of mother-child interaction that predict social behavior and academic achievement in school.

The observational session concluded with the interviewer administering the final portion of the brief interview to the mother, which included questions about use of child care for the focal child; the mother’s participation in educational, training, and employment activities; and the family’s residential situation. The interviewer also completed ratings about the home environment and about the observational session (for example, whether there had been others in the home during the session and whether the session had been interrupted), and the mother completed a brief "self-administered questionnaire" with items concerning her subjective sense of well-being, perceptions of the focal child, and reactions to the mother-child interaction tasks.

The initial instructions to the mother as well as the series of mother-child interaction tasks (but not the interview segments of the observational session) were videotaped. The videotapes of mother-child interaction were then coded in two independent research laboratories. The affective quality of mother-child interaction was rated by a team of researchers at the University of Minnesota. Ratings of mother-child interactions related to the emergence of literacy were made by a team of researchers at Harvard University, based on transcriptions of verbal interactions during the book reading and wheels tasks and on the interactive behavior from the videotapes for these tasks.

The families in the New Chance Observational Study also participated in the data collection that was part of the full New Chance Demonstration. Thus, there is information on the families prior to their being randomly assigned to the evaluation’s experimental or control groups ("baseline data"), before any program effects could have occurred. There are also data from the 18- and 42-month follow-up surveys, both of which included measures of the mothers’ psychological well-being; educational attainment, employment, earnings, and welfare receipt; residential situation; use of child care for the focal child; and fertility.

The surveys from the full evaluation also included interview-based measures of parenting. The analyses reported on in this monograph focus on parenting measures from the 18-month follow-up survey, because these were collected close in time to the observational measures. The 18-month follow-up included three parenting scales based on maternal report: Warmth, Control, and Parenting Stress. The 18-month follow-up also included a measure of the emotional support and cognitive stimulation available to the child in the home environment, based on a combination of questions asked of the mother and ratings completed by the interviewer. This measure, the Home Observation for Measurement of the Environment-Short Form (or HOME-SF), provided a total score as well as Emotional Support, Cognitive Stimulation, Harsh Discipline, and Physical Environment subscale scores.

Table 1 lists the parenting measures available for the sample of the New Chance Observational Study and the source of each measure. It is important to note that the parenting measures included in the New Chance Observational Study rely on several different informants (mothers, interviewers, and coders of the observational session videotapes). Thus, we are not confined to one data source in examining the New Chance Program’s possible impacts on parenting behavior. Each of the measures (along with information about its previous use and psychometric properties) is described in detail in the monograph.

The focal children’s development was assessed as part of the 42-month follow-up. Direct assessments of their cognitive development were carried out using the School Readiness Component of the Bracken Basic Concept Scale, which assesses children’s knowledge of such concepts as colors, letters, numbers, shapes, counting, and making comparisons. In addition, mothers rated their children’s behavior problems (using the Behavior Problems Index) and positive social behaviors (using the Positive Behavior Index) and responded to questions concerning their children’s health. The Behavior Problems Index provides a total score and subscale scores for behaviors that reflect antisocial, anxious/depressed, headstrong, hyperactive, dependent, and peer conflict/withdrawal behaviors. The Positive Behavior Index provides a total score and subscale scores for compliance, social competence, and autonomy.

For those children already in a formal child care situation or school setting, teachers were asked to complete a survey, in which they rated children’s academic and behavioral adjustment to school. They also completed the Behavior Problems Index and the Positive Behavior Index.

Some of the parenting measures used in the New Chance Observational Study had been used in other studies of high-risk families. On these measures, New Chance mothers had less positive scores than mothers in the other high-risk samples. When reading to their children, New Chance mothers showed lower frequencies of "Nonimmediate Utterances," that is, speech that extends beyond the information in the book (the specific words in the text and the pictures shown) to make connections with other experiences and information. Previous research shows Nonimmediate Utterances to be important in laying the groundwork for later literacy in children. Mothers in the New Chance Observational Study also expressed hostility to their children more often than the mothers in another high-risk sample. Harsh mother-child interaction is important to development, predicting less positive adjustment in children.

Among mothers in the New Chance Observational Study sample, more emotionally supportive and cognitively stimulating parenting behaviors were associated with higher maternal literacy, more educational attainment, better maternal psychological well-being, greater social support, and the child’s having participated in child care.

Time of Collection in New Chance Study
Type of Parenting Measure	Variables	Observational Session	18-Month Interview	21-Month Interview	Chapter

Observational Measures of Mother-Child Interaction Related to Literacy	Book Reading Task: Total Number of Utterances Number of Nonimmediate Utterances Percentage of Immediate Utterances Number of Discussion Topics Book Reading Quality Wheels Task: Objects Named Objects/Elicitations Mother’s Ease of Ideas	Derived from two tasks only: book reading task and wheels task			5

Observational Measures of Affective Quality of Mother-Child Interaction	Ratings of Mother: Supportive Presence Intrusiveness Hostility Quality of Instruction Confidence Harsh Treatment Ratings of Child: Persistence Enthusiasm Negativity Compliance Experience of Session Affection to Mother Avoidance of Mother Ratings of Dyadic Behavior: Quality of Relationship Boundary Dissolution	Derived from full videotape			4

Combination of Interviewer Ratings and Maternal Report: Home Observation for Measurement of the Environment — Short Form (HOME-SF)	Emotional Support Cognitive Stimulation Physical Environment Harsh Discipline HOME Total		X		6

Maternal Report Scales	Warmth Control Stress		X X X		6

	Overall Parenting Time Parenting Chore Time			X X

Positive program impacts, although modest in magnitude, were seen in both the affective quality of mother-child interaction and the cognitive stimulation that the mothers provided. In terms of the affective quality of interaction, mothers in the experimental group had significantly lower scores on the Harsh Treatment measure, higher scores on the HOME-SF Emotional Support subscale, and higher scores on the Maternal Warmth scale. In terms of cognitive stimulation, mothers in the experimental group received higher scores on the Book Reading Quality measure. Positive program impacts were also found on the HOME-SF total score and on time use measures devoted to parenting (both Overall Parenting Time and Parenting Chore Time — time spent engaged in such chores as feeding and bathing children).

Differences were found for parenting measures based on direct observation of mother-child interaction ( Harsh Treatment and Book Reading Quality), measures based entirely on maternal report (Warmth, Overall Parenting Time, and Parenting Chore Time), and measures that rely on a combination of maternal report and interviewer ratings (the HOME-SF total score and Emotional Support subscale).

Positive implications of supportive and stimulating parenting behavior combined with the negative implications of maternal psychological distress and stress in the larger social context to shape the developmental outcomes of children in this sample.

Given the high levels of maternal psychological distress and stress in this sample, bringing about positive impacts on child outcomes might have required (1) a more intensive "dosage" of parenting education classes or other program components that enhance parenting behavior, (2) sufficiently intense program components directly focusing on the mothers’ psychological well-being and living situations, and/or (3) high-quality child care, with children participating over a sustained period of time.

Observational measures increase one’s confidence in findings on program impacts because the coding is carried out in an extremely rigorous way. They also tap into certain behaviors that are important to children’s development, but that mothers may not be able to report on (such as their use of Nonimmediate Utterances during book reading). Also, the findings indicate that observational measures have advantages over interview-based measures for predicting variation in children’s scores on specific child outcomes. For example, consideration of the observational measures of mother-child interaction significantly improved prediction of the Behavior Problems Index (as reported on by the mother) even when parenting measures based on interviews had already been taken into account. In short, observational measures provide valuable information that can enrich the evaluation of programs such as New Chance.

The observational study interaction tasks, and the coding of behavior from those tasks, grew out of previous research in the laboratories of Catherine Snow, Jeanne De Temple, and their colleagues at Harvard University, and Byron Egeland, Nancy Weinfield, John Ogawa, and their colleagues at the University of Minnesota. Since the measures have been refined and improved over time, it is generally not possible to compare directly the findings from the New Chance Observational Study with those from previous studies. For a few specific measures, however, modifications in rating scales have not occurred. When we look at findings for those measures that can be compared across studies, we see that New Chance Observational Study families are at particularly high risk in terms of parenting behavior.

One key measure of mother-child interaction related to the development of children’s literacy is the proportion of maternal talk during the book reading task that involves Nonimmediate Utterances, that is, connecting the story and pictures to other events, people, and objects. Such talk also involves making predictions, asking for inferences, and providing explanations. Previous research using this measure in the Home-School Study of Language and Literacy Development, a longitudinal study of low-income mothers and their children, found maternal use of Nonimmediate Utterances to be related to important child outcomes in the early years of elementary school, especially the children’s use of language and their literacy skills.

Nonimmediate Utterances in the context of a book reading task constituted 10.5 percent of mothers’ utterances in the Home-School Study of Language and Literacy Development, but only 3.5 percent of talk among New Chance Observational Study mothers. The researchers note that "Since we have found that maternal use of Nonimmediate Talk relates to later child outcomes . . . the very low proportion of Nonimmediate Talk produced by the New Chance mothers is troubling."

The ratings scales of the affective quality of mother-child interaction used in the New Chance Observational Study are adaptations of scales developed in the Minnesota Mother-Child Project, a longitudinal study of high-risk mothers and children. The Hostility rating scale has not been substantially modified for the present study, and results can be compared across studies. Ratings of 5 or above indicate that a mother is more hostile than not in interacting with her child during the course of the mother-child tasks. Seven percent of the New Chance Observational Study sample scored 5 or above compared with 2.7 percent on the Minnesota Mother-Child Project sample.

These findings suggest that the New Chance Observational Study sample is at greater risk in terms of parenting behavior than the previously studied high-risk samples, underscoring the importance of attempting to enhance parenting behavior in the population of young single mothers in poverty.

Family characteristics were significantly linked with parenting behaviors, and associations were more consistent with 18-month than baseline variables, suggesting that the more current family context is of greater importance to parenting behaviors. These characteristics included social support; measures of the mother’s psychological well-being, residence pattern, maternal education, and literacy; and the child’s participation in child care during the initial 18-month follow-up period.

For example, mothers who reported more sources of social support at the time of the 18-month follow-up interview had more positive relations with their children in terms of the observed affective quality of mother-child interaction, the observed literacy-related aspects of interaction, and the harshness of discipline as measured by the HOME-SF. Mothers at high risk for depression at the 18-month follow-up had lower scores on all but one of the HOME-SF subscales, had lower observed Book Reading Quality scores, reported more parenting stress, and described themselves as using more controlling disciplinary practices.

Significant positive program impacts on parenting behavior were found on a range of parenting measures:

Positive program impacts occur in aspects of parenting that have previously been identified as particularly important to the development of children in poverty. For example, harsh parent-child interaction has been found to occur with greater frequency among families experiencing economic hardship. Harsh parent-child interaction, in turn, is a key contributor to the less favorable adjustment of children in poverty.

In general, the findings indicate that the New Chance Program had positive impacts on parenting behaviors important to development. There are three caveats, however. First, there were differences on only a minority of the parenting measures examined. Second, all but one of the program impacts, that on the HOME-SF Emotional Support subscale, were small in magnitude ("effect size"). Finally, a single significant finding ran counter to this pattern of positive impacts: Mothers in the experimental group received lower Ease of Ideas scores, observed during the mother-child task that called for eliciting from the child the names of objects that have wheels. That is, mothers in the experimental group were observed to be less facile in coming up with hints and clues for the child. This difference, however, was found to be attributable entirely to a larger number of mothers within the experimental group who did not grasp the goal of this task at all. Thus, this group difference might just as easily be interpreted as an indication of task or test anxiety, or of problems with the interviewers’ explanations of this task, than as a reflection on the quality of the mothers’ parenting behavior.

The research team concluded that the New Chance Program had positive, albeit modest, impacts on parenting behavior.

Among mothers in the experimental group within the New Chance Observational Study sample, greater participation in parenting education classes was related to more positive parenting behavior. Interestingly, however, this pattern was also found for participation in other New Chance Program components.

We looked at four aspects of experimental group mothers’ participation in New Chance: (1) participation in parenting education classes, (2) participation in a broader set of program activities that addressed parenting behavior in some way, (3) participation in human capital development components of the program (that is, components intended to improve the mothers’ ability to obtain and keep a job, including adult education and employability development classes), and (4) total program participation.

Even after controlling for baseline characteristics that predicted each of these aspects of program participation, certain parenting measures continued to be significantly associated with the extent of program participation among mothers in the experimental group. Thus, within this group, scores on the HOME-SF Emotional Support subscale continued to differ significantly according to extent of participation as defined in each of the four ways; and the observational measure of Book Reading Quality differed according to participation in the human capital development components of the program as well as in the program overall.

It should be noted that characteristics we did not have baseline measures of, and thus could not control for, might be linked both to greater program participation and to parenting behavior, and these undocumented variables might help account for the associations we have noted. These findings nevertheless raise the possibility that components of programs for young mothers in poverty including but going beyond parenting education may have positive implications for parenting behavior. Further study is needed to explore how and why program components directed at mothers’ education or employment skills, and their overall program participation, might affect their parenting behavior.

The findings on child outcomes in the New Chance Demonstration present us with a paradox. Although there were positive impacts on measures of parenting, children in the experimental group did not do better in terms of their cognitive and social development when these were assessed at the 42-month follow-up. On most measures, there were no program impacts. However, on some measures there were unanticipated negative program impacts. For example, mothers in the experimental group described their children as having less positive social behavior. This pattern held in both the full demonstration study sample and the smaller observational study sample.

Clearly, we need to consider factors other than parenting behavior as contributors to children’s development — for example, mothers’ psychological well-being, family economic status, and the experiences of mothers and children outside the mother-child relationship. New Chance appears to have had unexpected negative program impacts on some of these further factors. For example, mothers in the experimental group in the observational study sample were more, rather than less, depressed at the time of the first (18-month) follow-up survey. Mothers in the experimental group of the observational study sample also reported less life satisfaction.

We considered both the role of parenting behavior and variables that measured other important aspects of the mothers’ and children’s lives as predictors of five selected child outcomes within the observational study sample (children’s total scores on the School Readiness Component of the Bracken Basic Concept Scale, the Behavior Problems Index total score as reported by both mother and teacher, and the Positive Behavior Index total score as reported by both mother and teacher). We confirmed that when measures of parenting behavior were taken into account, our ability to predict children’s scores on several of these outcomes improved significantly (specifically, the maternal report of the Behavior Problems Index and Positive Behavior Index and the teacher report of the Positive Behavior Index).

More supportive and stimulating parenting thus predicted more optimal developmental outcomes on specific measures of development. However, in addition, we found that variables reflecting maternal psychological well-being (for example, measures of life satisfaction and of aggravation and stress in parenting) and variables reflecting the larger social context of the families (for example, measures of difficult life circumstances and of number of changes of residence since enrolling in the evaluation) were also significant predictors of several child outcomes. Greater maternal psychological distress and greater stress in the larger social context predicted less positive developmental outcomes.

An important finding of this study, then, is that while parenting behavior was a significant predictor of specific child outcomes, it was not the only predictor. The positive influence of supportive and stimulating parenting behavior combined with negative influences of psychological distress in the mother and stress in the family’s larger social environment. Children’s developmental outcomes reflect influences not only from within but also outside the mother-child dyad.

Outcomes for children in the context of an intervention such as New Chance might improve if the intervention directly addressed these problems through more intensive mental health intervention for the mothers; if the "dosage" of program components with positive implications for children’s development, such as parenting education, were substantially increased; or if program elements targeted to the children themselves were strengthened. Direct observations of New Chance child care settings placed these child care centers just below the "good" range in terms of quality. For positive child outcomes to occur, children from high-risk families may need higher-quality care. In addition, beyond the first program follow-up (at 18 months), children in the New Chance experimental group were not found to participate in more child care than those in the control group. Indeed, the increased child care participation of children in the experimental group tended to occur only during the first months of their mothers’ program participation, when the mothers were engaged in on-site classes and activities. Positive program impacts on children may require sustained participation in high-quality child care.

Measures of parenting based on direct observation of mother-child interaction contribute to our understanding of the New Chance Program in several ways. First, these measures increase our certainty about program impacts on parenting. If positive impacts on parenting were found only for maternal report measures, we might question them on the grounds that mothers in the experimental group (aware that they had access to New Chance Program services and that outcomes of the program were being assessed) could report more favorably because they felt it to be expected of them. Even interviewers aware of which research group a family was in could be subject to such "response biases." Coders of the videotapes were unaware of which research group a family was in, yet positive program impacts were found on observational measures.

Second, observational measures provided information that was different from and complementary to that provided by interview-based measures of parenting. For example, a positive program impact was found on the observational measure of Book Reading Quality, yet no program impacts were found on the interview-based measure of cognitive stimulation (the HOME-SF Cognitive Stimulation subscale). The observational measure looks at the nature or quality of mother-child interaction in a book reading context, for example, the mother’s fluency, intonation, and comfort level in reading the book to her child. By contrast, the HOME-SF measure of cognitive stimulation assesses the quantity of literacy-related and other stimulating activities: for example, how many books were in the home, and how often the mother reads to the child. Without a measure of the nature or quality of cognitive stimulation, we might not have known that there were program impacts in this area. Correlations between observational and interview-based measures of parenting confirm that these measures are related but do not substantially overlap.

Finally, we found the observational measures to be helpful in predicting child outcomes. When we distinguished among maternal report scales, HOME-SF subscales, and observational measures as predictors of the selected child outcomes, we found that the HOME subscales and observational measures were generally better predictors than the maternal report scales. We also asked whether observational measures added to our ability to predict variation in children’s developmental outcomes after the maternal report scales and HOME-SF subscales were already taken into account. We found that for two of the five selected child outcomes, observational measures added significantly to our ability to predict child outcomes even when the other parenting measures had been taken into consideration.

The fact that the observational measures do not rely at all on maternal report also helps eliminate the possibility that associations between parenting behavior and those child outcomes based on maternal report (for example, the Behavior Problems Index, as reported by the mother) are not merely a reflection of common attitudinal or reporting tendencies across different sets of maternal report measures.

Thus, observational measures of parenting behavior provide different information than that available through interview-based measures of parenting, diminish concerns about possible response biases and correlated measurement error, and add significantly to the ability to predict specific child outcomes. Where the examination of parenting behavior is a high priority, observational measures of parenting add substantially to the strength of an evaluation.

The New Chance Observational Study lies at the confluence of rising interest in policy-relevant research among developmental psychologists, interventions focusing on two generations in a family, and increasing demand for nontraditional forms of survey research. In attempting to increase the size and representativeness of samples, some developmentalists are departing from the familiar model of laboratory-based research, sometimes supplemented with home visits, to undertake their data collection through contracted survey research. Survey research on parenting and child development has traditionally involved querying parents about their parenting practices and their children’s development. In recent years, survey interviewers have also been asked to rate parent-child interactions and the home environment, and, in a few cases, to administer structured parent-child interactions similar to those usually carried out in the child development laboratory. New Chance, while not alone in its use of survey methods to study child development, is one of the few studies to use survey interviewers to conduct observational work with mothers and children. Within this context, Part II of the monograph seeks to familiarize readers with the "survey model," document how the observational study was conducted, assess the success of the effort, and consider both specific recommendations for future work and broader implications for research design.

This study affords a unique opportunity for examining methodological issues in the measurement of parenting and child outcomes for two reasons. First, it taps a diverse set of measures and data sources: taped mother-child interactions coded under rigorous conditions in university laboratories, self-reports of mothers, and ratings by survey interviewers of mother-child interactions and the home environment. Just as important as the rich variety of domains tapped is the fact that the study was conducted within the framework of a survey research model, while "stretching" this model to exploit it in innovative ways.

Chapter 10 of the monograph seeks to enhance the value and accessibility of the survey field to social scientists who have not made survey research the primary focus of their careers and to begin to bridge the communication gap between survey researchers and their colleagues — even from the same disciplines — who may emphasize a substantive research agenda more than a method. Without a full appreciation of the constraints of the survey model, researchers who commission survey work may have difficulty managing it and even find themselves disappointed with the results. At the same time, they will be shortchanged if they look to the field primarily for a data collection capability and fail to heed its lessons on survey-based measurement.

In contrast to an academically oriented model, survey research is built on a division of labor between those responsible for the conceptual and analytical aspects of research and those who actually collect the data. The survey model places responsibility for data collection in the hands of "distant proxies" for "absent researchers." These proxies, the survey interviewers who collect data on surveys ranging from small local studies to large recurring government surveys, are neither assumed to possess nor expected to master a conceptual appreciation of the research in which they participate. For these reasons, survey researchers provide interviewers with explicit rules to direct them through the data collection — rules that, in theory at least, require minimal judgment to apply. The resulting survey model is characterized by an orientation toward production, a contrived and stylized format for the interaction of interviewer and respondent, and precise programming and standardization of interviewer behaviors. Designing sound data collection instruments and procedures involves accommodating to these three aspects of the survey model.

The constraints, or "rules," of the survey model may be viewed as potential limitations in conducting social science research, but the survey community’s trove of research on survey measurement can be viewed as a valuable resource. The underlying focus of most of this work is the reduction of error in surveys.

The bulk of the measurement literature has focused on the design of measurement instruments, that is, questionnaires, and less on other aspects of measurement, such as interviewer or mode effects. If this literature yields an overarching lesson, it is a humbling one: that there are many, many features of questions and questionnaires that can affect survey response. An awareness of such measurement threats is helpful both in developing measures and related data collection instruments and in assessing their likely strengths and limitations.

The survey literature also speaks to the ways in which interviewers may affect measurement in a study like this. Evidence for the prominence of interviewer effects in situations requiring more judgment or unprogrammed behavior is especially relevant to the present work, which required interviewers to make substantive ratings of the home environment and mother-child interaction, obtain time-use information through a series of open-ended probes, and administer a scripted observational protocol, while applying general principles to unscripted situations.

Given the importance of interviewers and the expansion of their responsibilities with ever more challenging studies, focusing on the cognitive demands of their tasks will help in assessing what is sensible to ask them to do and helping them to do it. The last decade or so has seen the emergence of a broader interest in cognitive aspects of survey response, beyond simply the recall of information. This interest has been directed primarily toward the tasks facing respondents during an interview, such as the strategies they use to estimate and report the frequency of a particular behavior. The cognitive framework can also be applied to the cognitive tasks faced by interviewers during a survey interaction, by examining, for example, the ability of interviewers to process the information necessary for making substantive ratings at the same time they are reading interview questions and recording answers. Viewing the demands of the survey interaction on both respondents and interviewers from a cognitive perspective provides a helpful framework for integrating what is known about the instruments used to collect data, the interviewers who administer them, and the respondents who provide the data.

As discussed in Chapter 11, and summarized below, implementing the observational research that is the central subject of this monograph involved both the design of data collection instruments and procedures and the data collection effort itself. In carrying out these tasks, we confronted and sought to address a host of survey measurement issues, most notably the overarching issue of adherence to the goals and objectives of the "absent researchers" who place their research in the hands of survey interviewers. Our approach included the following steps.

Survey interviewers are not expected to have the appreciation of a study’s theoretical underpinnings that would allow them to work from only a researcher’s semistructured outline. Therefore, the strategy for realizing the objectives of the "absent researchers" was to program interviewer behavior in the observational session carefully. This meant providing interviewers with a data collection instrument resembling a survey questionnaire, which specified the precise language to be used and the actions to be taken. The instrument included detailed instructions for presenting, arranging, and withdrawing the various props used in the tasks (book, games, and gift) and for coordinating management of the props with the script. Also included were language and instructions for verifying that the mothers understood the objectives of the task, decision rules for determining how much time to spend on each, and, for one task, decision rules as to whether a simple or complex version, or both versions, of the task was to be administered.