Speed or Safety? Science publishing in the time of COVID-19

Global pandemics demand fast, evidence-based responses. This poses a conundrum. Communication of scientific research is deliberately and excruciatingly slow. After an article is submitted to a journal of choice, rejected, and re-submitted several times, it is sent to a group of peer reviewers, where it probably sits untouched on a desk for a number of weeks before being evaluated. The full submission-to-publication timescale is about nine months – clearly, this is not an option in the current climate. Science must be rapidly disseminated to guide our response to the outbreak. And although preprint servers – platforms where scientists can post full drafts of papers (preprints) that skip the formal peer-review process – have been endorsed as a solution, there are concerns about their communication to non-scientific audiences that need to be addressed.

Preprints make research available at least three months prior to journal publication. They are becoming increasingly popular; in the four months following the first COVID-19 case, at least 37.5% of all COVID-19 related articles were hosted by bioRxiv and medRxiv, the two preprint servers for biomedical research.

At first glance, this can only be beneficial. Preprint servers host a remarkable diversity of institutions. By allowing research groups to access and build upon one another’s preliminary results, they ensure that the scientific community is using its resources effectively, and avoiding simultaneous duplication of work. These global collaborations are the only way research can keep up with the virus – but also remain neutral, objective and statistically valid, in compliance with the standards of the scientific method.

Problems arise when preprints are prematurely broadcast outside the scientific community. Scientists consider themselves to be writing for other scientists, who treat preprints as what they are; unfinished products (in the sense that they have not been checked outside of the research group). But preprints are available to all. And in a pandemic, the audience of scientific articles grows dramatically – COVID-19 preprints are accessed and distributed at least 15 times more than non-COVID-19 preprints. Open access has hundreds of benefits; dropping paywalls increases scientific participation in developing countries, for example. However, making a draft freely available is dangerous. It allows politicians and the media to irresponsibly overhype information which is, on the whole, speculative. And it risks damaging public trust in scientists when they inevitably get things wrong.

In January, a preprint by scientists in Delhi appeared on bioRxiv, discussing resemblances between the coronavirus and HIV (the virus that causes AIDS). The ambiguous phrasing of its abstract, though (“uncanny similarity … unlikely to be fortuitous”), suggested that the coronavirus had somehow been engineered by humans – and fuelled a spread of Twitter conspiracy theories that COVID-19 was a laboratory accident or, worse, a bioweapon. Luckily, the study was withdrawn before any news outlet could cover it – since the world of preprints is relatively new, and many are still sceptical, the retraction of faulty work is fast. But these responses cannot be relied upon indefinitely.

So, it is the overhyping of preprints, not the existence of them, that has the potential to influence behaviour and endanger public health. How can this be avoided? A small yellow banner on the screen reminding readers that preprints aren’t conclusive is not enough; neither is a comments section, which journalists and communicators almost certainly do not read. Either preprints should be available only to scientists, or their use should be made safer for non-scientific audiences. One route could be to require authors to include a plain-language summary, addressing the specific limitations of their research, as part of the preprint submission process. This way, we would see the collaborative benefits of preprints without misleading the public.

However, just because a study has been peer-reviewed doesn’t necessarily mean it should form the basis of public policy. Peer-review has its own problems; aside from being slow, it’s often skimpy, and authors can have competing interests. Indeed – most retractions of COVID-19 papers have been from not preprint servers, but high-profile medical journals – with severe consequences.

The Trump administration’s decision not to accept COVID-19 testing kits from overseas in March was based on a single, unreplicated study which claimed that “50 percent or 47 percent” of positives were false. This would have been bad enough, but the study had actually been retracted, for unknown reasons, just a few days after its publication in a Chinese journal. In other words – at a critical time in the early development of the pandemic, an entire continent may have been denied access to World Health Organisation-approved tests, because of findings that weren’t even trusted enough to remain in the scientific literature.

Another paper published in Annals of Internal Medicine in April concluded, alarmingly, that neither surgical nor cotton masks were effective in preventing the spread of COVID-19. Before it was retracted, this study was cited by over 100 news outlets, almost 10,000 Twitter users, and the World Health Organisation. Not only did it only have four participants, it actually went on to describe an 80.4% reduction in viral emission by cotton masks (how this was classed as “ineffective” is baffling, given that even N95 respirators block out 95% of the virus).

It is evident, from the fact that formally peer-reviewed studies could be this misleading, that the main issue with the COVID-19 publishing landscape is not peer-review status, but fast and careless distribution – an issue that is not unique to preprints.

Even in the face of a pandemic, we cannot lose sight of the fundamental aim of scientific research: to lower uncertainty and provide healthcare systems, policy-makers and the public with the necessary information to improve individual and public health. The collective effect of too much hasty, unvetted research is that science loses its credibility. Herbert Holden Thorp, editor-in-chief of Science, begs scientists to underpromise and overdeliver – to structure COVID-19 papers with a larger and more diverse audience in mind, and be clearer about the limitations of their work. We are moving towards an open-access, preprint-first world, and effective communication would ensure we all benefit from it.

Artwork by Arpita Chatterjee