When evidence leads to policy, but implementation falls short
A promising education programme in the Dominican Republic had full government backing—until it hit the real world. When it transitioned from a randomised trial to nationwide implementation, actual classroom delivery collapsed from 83% to 37%.
The country’s Ministry of Education was trying to address a persistent problem: too many young people were leaving school before completing secondary education. Earlier research suggested that students underestimated the benefits of staying in school, so the Ministry worked with researchers to develop Aprendiendo el Valor de la Educación (AVE), or Learning the Value of Education.
Students watched short videos about the economic, social, and personal benefits of education, followed by discussions with their teachers. A randomised trial found that AVE improved test scores and reduced dropout rates. On the strength of those results, the Ministry adopted the programme nationwide and included it in the official school calendar.
But implementation proved more difficult.
Researchers wanted to understand what was getting in the way. They tested several approaches: showing school principals evidence that the programme improved student outcomes, offering a modest financial incentive, and providing additional implementation assistance. None significantly increased compliance. Ironically, two simple reminder calls did, increasing implementation by 20 percentage points.
This gap is far from unique to the Dominican Republic. EEF research shows that effect sizes often drop when scaling an intervention from an efficacy trial to an effectiveness trial. For example, an approach delivered directly by experts might yield high gains, but when handed over to everyday teachers and third-party trainers, the practical challenges of the school day can alter results. This evaluation pipeline ensures that only approaches proving scalable and impactful in real-world settings are recommended for wider rollout.
The findings draw attention to an important but sometimes overlooked part of evidence-informed decision-making. Generating evidence about what works is one step. Getting that evidence into policy is another. What happens after the policy decision matters just as much.
In this case, the challenge did not appear to be simply whether principals understood or believed the evidence. Those who believed the programme would benefit students were no more likely to implement it. Instead, the researchers found indications that competing responsibilities and limited attention contributed to the implementation gap.
The lesson is not that the intervention stopped working at scale. Rather, evidence of effectiveness does not automatically tell us what it will take to deliver an intervention through routine, everyday systems.
For the evidence community, that raises an important question: once evidence has informed a policy decision, are we paying enough attention to the institutional conditions needed to turn that decision into practice?
