No Validated Instrument Exists to Predict 'Voltage Drop' Before Governments Scale an Education Pilot Nationally
Problem Definition
85-years-of-learning figure as if it were a property of the intervention rather than a property of a specific delivery channel: there is no standardized, validated instrument that predicts, before a government scales a pilot nationally, how much of the effect size will survive the handoff from NGO-run implementation to routine bureaucratic delivery.
's Kenya teacher-contract experiment, the Action Schools! BC cluster-RCT in Canada, decades of "voltage drop" anecdotes across global health and education — that all document the same pattern after the fact.
What we do not have is a leading indicator that a finance ministry or a donor can check before signing a scale-up budget: something equivalent to a stress test that says "this program's effect depends on implementer-level monitoring intensity X, your civil service currently delivers at intensity Y, and Y is below the threshold where the effect held in trials."
Instead the field relies on qualitative "readiness assessments" that are rarely pre-registered, rarely validated against actual scale-up outcomes, and almost never publicly scored the way effect sizes themselves are.
Implementation science has produced frameworks — RE-AIM, the Consolidated Framework for Implementation Research — that are good at describing after the fact why something degraded, and much weaker at forecasting the magnitude of degradation before money moves.
J-PAL and its partners have been unusually honest that TaRL's own scale-up required years of iterative adaptation with Pratham still embedded in delivery; that adaptation process is exactly the tacit knowledge that a standardized instrument would need to capture and that currently lives in program officers' heads rather than in a replicable protocol.
The cost of not solving this is not abstract.
Every government that scales an education, health, or cash-transfer pilot on the strength of an RCT effect size and then gets the Kenya result — a measured near-zero — has burned a budget cycle, a political mandate for "evidence-based policy," and, in the education case, a cohort of children who did not get the learning gains the citation promised them.
Given that roughly 70% of 10-year-olds in low- and middle-income countries were estimated to be below basic reading proficiency post-pandemic per the World Bank's learning poverty metric, the stakes of getting scale-up prediction wrong are not a rounding error, they are the difference between hitting or missing foundational learning targets for hundreds of millions of children.
Root Causes
Implementation-science frameworks (RE-AIM, CFIR) are diagnostic and retrospective, built to explain degradation after it happens rather than to forecast its magnitude before a scale-up budget is committed.
The tacit adaptation knowledge that let programs like TaRL survive scale-up (iterative redesign, continued NGO co-implementation) lives in program officers' institutional memory, not in a documented, transferable protocol other ministries can apply ex ante.
Donor and government incentives reward citing the original trial's effect size to justify funding, creating no institutional demand for a sobering pre-scale-up fidelity forecast that would lower the projected benefit-cost ratio.
No shared, cross-sector dataset links pre-scale-up institutional-capacity indicators (civil-service monitoring intensity, teacher/staff turnover, hiring authority) to the actual post-scale-up effect-size retention, so there is no empirical base from which to build or validate a predictive tool.
Scope
Discussion
Discussion (1)
We are effectively treating institutional implementation capacity as a constant rather than a variable, which is why we keep mistaking pilot-phase efficacy for inherent program DNA. If we refuse to build a "fidelity-loss diagnostic" into the pre-scale assessment, aren't we just setting ministries up for inevitable policy failure?
