Part I of this two-part post discussed MDRC’s work with practitioners to construct valid and reliable measures of implementation fidelity to an early childhood curriculum. Part II examines how those data can reveal associations between levels of fidelity and gains in children’s academic skills.
The Every Student Succeeds Act (ESSA) requires school districts to implement a five-measure accountability system. Thirty-five states have ESSA accountability systems that include measures of career readiness. Here is one example of how a school district might effectively strengthen students’ career readiness using Career Academies.
Lessons from the Grameen America Evaluation
In any study, there is a tension between research and program needs. This program’s group-based microloan model presented particular challenges for random assignment. Reflections in Methodology looks at how the research design was adapted to allow a fair test of the program’s effectiveness without hampering its ability to operate.
As an alternative to random assignment, a regression discontinuity design takes advantage of situations where program eligibility is determined by whether a score exceeds a threshold. With careful attention to assumptions, analysis, and interpretation, this quasi-experimental design can provide rigorous estimates of program effects. Reflections on Methodology outlines some considerations.
Schools use individual screening tests to identify students at risk of falling behind in their reading levels. Could predictive analytics, incorporating multiple composite and subsection scores from a series of tests over time, do a better job of identifying at-risk students? Reflections on Methodology gives an example of this approach.
Lessons from the Grameen America Formative Evaluation
Random assignment is prized for its rigor, but it’s not always feasible to carry out. This Reflections in Methodology post outlines other strong options for studying the effects of a program and illustrates the application of some key considerations in a specific context.
Observation tools allow researchers to rate practitioners’ use of a program or curriculum according to the time spent on its components and how they were implemented. Reflections on Methodology explains how researchers and model developers can collaborate to develop useful assessment tools and the benefits and challenges involved.
In NYC P-TECH Grades 9-14 schools, students take an integrated sequence of high school and college courses with the goal of completing both high school and college, while simultaneously being exposed to hands-on work experiences. This infographic describes the model and introduces MDRC’s evaluation of it.
The new book Randomistas describes how randomized controlled trials (RCTs) have revolutionized many fields. RCTs are a uniquely powerful tool, but they are not the only way to build knowledge about effective programs for low-income people.
By combining prior beliefs about a program’s effectiveness with new data to produce a distribution of impacts, Bayesian statistics provides an alternative to classical methods that may be more useful for policymaking. Reflections on Methodology discusses some issues with and applications of this approach.