Ryan C Briggs

Randomized control trials (RCTs) are the gold standard for some kinds of development interventions, but they are naturally limited in how broadly they can generalize to new contexts. This point was recently raised in an excellent debate and discussion between Abhijit Banerjee and Angus Deaton.

I mentioned this before and drew a distinction between doing RCTs with people and doing RCTs with larger social units within countries. Countries are far more heterogeneous than people, and so it is easier to generalize from some sample of people to the rest of humanity than it is to generalize from some sample of villages to all villages.

The reason I’m bringing this up again is that I didn’t realize how far medical science has moved towards increasing internal validity at the cost of external. From the Aid on the Edge of Chaos blog:

Ironically, “the canonical tenets of ‘scientific excellence’” are threatening to undermine the whole enterprise. One rather shocking – for me, at least – example relate to the latest developments in research on mice, where a lot of resources and funding have been poured into the cloning of genetically identical animals in order to enable fully controlled, replicable experiments and rigorous hypothesis-testing. However, the findings of this research have turned out to be useless when applied in the real world of diversity and change.

I recommend reading the whole article. It is sadly comforting to know that every science has these problems.