With the General Election fast approaching, science might not seem the most obvious issue to decide your vote. I’d disagree—and not just because I’m a self-interested nerd clamouring for funding (though, given the economic and social benefits of research, and how far we are behind other major economies, that is a legitimate worry). The three main parties’ manifestos should leave not only scientists, but all voters with a graver concern: that the scientific method has barely permeated politics.
The scandalous omission is that no-one has made a commitment to more randomised, controlled trials of social interventions. That may sound less like a scandal and more like an arcane statistical complaint, but its consequence is that we have literally no idea whether or not most policies are effective or ineffective, or even good or bad—indeed, we have absolutely no idea if they work at all.
The randomised, controlled trial, or RCT, has long been considered the gold standard in medical research. The first stage is to concoct an intervention: it could be a drug, an exercise regime, a method of teaching children to read, or a way to reduce injuries in pub brawls. Then, you get a statistically significant number of trial subjects: patients for medical trials, perhaps schools or areas of the country when considering social issues. You split your subjects into two groups, one of whom gets the intervention, and one of whom—the ‘control’ group—you give a sugar pill instead of the active drug, or leave them with whatever was the standard before. Critically, you have to split your subjects up randomly, so that no bias afflicts to which group they are assigned.
That’s all there is to it—get a big enough sample, randomise your controls, and you’ve got yourself a gold-standard trial which can tell you to a given degree of statistical confidence whether or not a drug, policing strategy or education reform delivers any real benefits.
What’s the big deal about randomisation? A common feature of a government ‘trial scheme’ is to implement it in the neediest areas first. It may seem rational, even ethical, to dole out help where it’s most needed…but to do so totally invalidates your results. To take a simple example, when things are at their worst they often ‘regress to the mean’: if you’re in the area of the country with the most drug abuse, ‘things can only get better’ as New Labour might have put it. And even if your glowing results are valid (and with a badly-designed trial you have literally no way of knowing), there is no evidence that they will be replicated when you roll out the scheme to less disadvantaged neighbourhoods nationwide.
A common criticism of randomisation is that it is itself unethical—only half of your sample is getting the brilliant new intervention, whilst the remainder struggle on with whatever came before. However, the opposite is surely true: until you’ve done a proper RCT you don’t know which is better, and so it could be that you’re replacing a better system with a worse one without knowing. With a trial, the worst case is that 50% of test subjects receive disbenefit, and everyone in future gets whichever intervention is best: if instead you rely on politicians’ guesswork, the risk is that every future child, patient or criminal may get the worse option, at least until the next administration alters the policy on a revised whim. If one of the methods, old or new, does turn out to be leagues behind the other, statisticians can spot this and abandon the trial for ethical reasons, as happens occasionally in medicine.
So what’s wrong with good old political nouse? Well, the human brain is ill-equipped to assess complex systems like societies, economies or human bodies. The history of medicine is littered with ‘obviously true’ results discredited utterly by proper trial. For example, it turns out that vitamin supplements have no measureable effect on life expectancy. Vitamins aren’t good for you? Who knew? Well, no-one, until they tried it. Imagine what universal political truths we could tear up by subjecting them to scientific scrutiny…and imagine what huge benefits to public health, wealth and wellbeing might be waiting to be unlocked by doing so.
RCTs are no panacea: they will never be able to inform macroeconomics, for example, and cannot assess anything where the goal is unquantifiable. Where they do work, though, they will save time, money and lives, and the only cost will be politicians’ hubris.
The problem, it seems, is that though implementing policy RCTs should be a no-brainer, no-one is offering to do it. Unsurprising, perhaps, in a parliament where 584 of 646 MPs claim to have no interest in science and technology. So can I offer any concrete voting advice beyond the tedious man-in-the-pub call to inaction that ‘they’re all as bad as each-other’? Well, if you look beyond the manifestos, there is a tiny glimmer of light at the end of the tunnel.
The Campaign for Science and Engineering sent a pre-election open letter to the leaders of the three main parties; Gordon Brown’s slightly tardy response is solid but contains no surprises, and David Cameron responded with a reassuring rehash of manifesto material, but Nick Clegg’s reply, in its final point, expressed his ‘hope that this…would usher in an era…[of] regular use of Randomised Controlled Trials in testing new social policy initiatives in those circumstances when the balance of evidence is not conclusive.’
This isn’t perfect: there’s no policy commitment, it wasn’t in the manifesto and, with ‘when the balance of evidence is not conclusive’, there is the distinct whiff of weasel words. It sounds like a get-out clause which leaves the veto with politicians, and it’s also not obvious how the evidence can be ‘conclusive’ before a proper trial has been conducted. However, even this promise-free hope is a cut above the other parties.
Clegg’s aspiration may not be a dealbreaker, but it’s certainly a consideration. We currently have no idea whether what we’re doing works in areas as diverse as primary education and criminal justice. Whoever you vote for, letting the scientific method into politics where it’s appropriate is a policy we should all support vocally.
