Alzheimer’s cure or incurable hype?

Yesterday many newspapers claimed a major breakthrough in dementia treatment: “Scientists create first drug to halt Alzheimer’s’” (The Times); “Scientists discover first drug to halt brain decline in Alzheimer’s” (The Sun). Is this true, or is it yet another exaggerated false hope for patients and their families?

Digging in the detail will reveal no clear evidence that the drug works at all. The main result of the trial in humans was null: no evidence of better outcomes for patients taking this drug (alongside any other medication they take) compared with the placebo (alongside any other medicine they take). So how did a null result become a massive news splash claiming the opposite?

The glimmer of hope – a phrase used by The Sun, to give them some credit – comes from 15% of the patients who were not taking any other treatments as well. These patients did show better outcomes, but this was compared to the controls overall, not compared to a matched sample. This result could be due to chance, since there are many possible ways the data might have been analysed once broken down into subgroups. In other words, there’s a multiple comparisons problem, as explained in a nice blog by Tom Chivers.

This glimmer of an effect for a subset of patients might be enough to justify the huge investment needed to run another trial, specifically investigating if the result is real or spurious. Indeed, this trial might even have started, according to a quote in Forbes Magazine.

But it certainly does not justify the headlines today. There is no known reason why the drug would work only on its own, and not alongside other drugs. The story is based on an announcement about a conference talk that took place yesterday, rather than a peer-reviewed paper.

So is the hype coming from journalists, academics or the drug company? The main source of the stories – and the hype – is a press release that was posted onto a newswire.

This is a very common way for science and health news to be generated, and it is also a common way for hype to arise. We have found in our research that much of the exaggeration you find in the news is already in the press releases that feed the news.

The press release comes from the company, but the company is relatively small and is founded by, and advised by, scientists. It is not a huge multinational. And there are very strong quotes from these scientists. So it is not a simple case of cautious scientist and corporate press office hyperbole.

It is, of course, very common for scientists to try to find positives from a ‘failed’ study, without thinking through the statistical implications. It is tempting enough for scientists who have simply invested a few months of their time and a few hundreds or thousands of someone else’s pounds.

Phase 3 drug trials are in a whole other league – this one seems to have cost about $135m from a quick look at their website.

But it is not just the scientists and the press release – journalists are trained to see through this kind of hype, so I am not sure why they didn’t in this case. Interestingly, the news stories in the US paint a different picture to those in the UK, as Ben Goldacre points out.

So what is the take-home message? The results of this trial are more likely to mean that this kind of drug is a dead-end, rather than a new cure that will halt dementia progression.

Always dig in the detail.

Professor Petroc Sumner receives funding from ESRC, BIAL foundation, Alcohol Research UK (ARUK), British Psychology Society, Wellcome Trust, BBSRC, WICN, Nuffield and Royal Society.