What Is Implementation Research?

Public housing project

 

Understanding How and Why Programs Work

Implementation research is an integral component of most MDRC studies. While our impact studies investigate the effects of a new program or policy on individuals’ outcomes, it is the work of implementation research to investigate how the program produces those impacts by understanding the services that are delivered to enrollees and the factors that influence their implementation.

Implementation research is typically a multi-method inquiry that uses both quantitative and qualitative data to assess the programs and policies that are the subject of an evaluation. Whether part of an experiment or a stand-alone study, implementation research at MDRC is grounded in a logic model, which expresses the overall theory of change for a program. The logic model outlines the resources necessary for a program, the population it expects to serve, the activities that comprise the program, the specific services to be delivered, and the expected outcomes for participants.

What kinds of questions does implementation research address?   

Implementation studies cover a diverse range of questions. A central goal is to describe the services that are delivered — who has enrolled in the study or program; the content, quantity, quality, and structure of services that the program intended to deliver; and the extent to which the target population received the intended service. Implementation studies often go further — for example, assessing the organizational context, the process by which staff were trained and supported in delivering the program, or the quality of therapeutic relationships between staff and participants. To accomplish these goals, implementation studies might field surveys, conduct interviews with staff or participants, or observe services as they are being delivered.   

Implementation studies increasingly inform program scale-up, helping policymakers and administrators understand the features of systems or organizations that are best suited to implementing a program as it expands. How do local programs adopt, adapt, and implement evidence-based models? What can we learn across studies and across different fields of education and social policy? Implementation studies often examine the fidelity of services actually provided to the intended service model, as well as the successes and challenges associated with delivering services as designed. When local organizations implement an existing service model, they may intentionally adapt it to fit their local context. Implementation studies can document the reasons for, types of, and results of such adaptations.

Why is implementation research important?

Innovative educational practices and social programs are complex processes typically embedded in larger systems and organizational contexts. Whether within a randomized control trial or a stand-alone inquiry about program implementation itself, MDRC views implementation research as fundamental to understanding programs’ effects and providing lessons to strengthen future public investments. In randomized trials, implementation studies lay the foundation for understanding the difference in services received by program and comparison groups — the “treatment differential” — and thus provide the context for interpreting the findings of an impact study. But perhaps more importantly, good implementation research can provide lessons to practitioners about the best way to run effective programs and advice to policymakers about how to improve existing interventions. Our implementation research informs decision-making by policymakers, program operators, and others who aim to fully understand complex interventions. 

Examples of MDRC’s implementation research: