Corona research: Is preprint publication cutting corners?

As early as March, the German public broadcaster NDR had 15 million people subscribing to its coronavirus podcast with virologist Christian Drosten from the Berlin Charite University hospital. In the era of COVID-19, politicians around the globe are listening more than ever to virologists, epidemiologists and infectious disease experts to draw guidance for action from the latest findings.

It seems to be a case of all eyes on science at present. Scientific research is becoming relevant to everyday life in a way that it has never seemed to be before. The general public and politicians are waiting with bated breath for the next discoveries, as every new scientific finding could potentially change the lives of millions of people.

As a result, scientists are working under immense pressure. They are investigating the different progressions of the disease, experimenting with drugs that are already approved and researching vaccines. And there is demand for their results to be published as quickly as possible. This is where problems arise: Before now, speed was not the top priority in science.

The abnormal new normality

Normally, studies are subject to a relatively time-consuming scientific peer review process in which a body of experts from various disciplines examines the research results. This committee, consisting of people who are not associated with the study, is appointed by the journal to which the scientists submit their study for publication.

These experts review, comment and give feedback to the study authors. The authors then make adjustments and resubmit the paper. All this takes time; if things go badly, it can even take years. That is far too long when the research concerns SARS-CoV-2 and the lung disease COVID-19 caused by the virus.

Social media for scientists

Thanks to preprint servers like bioRxiv and medRxiv, the publication of potentially groundbreaking study results has become much faster.

Here, the entire peer review process is omitted. Anyone who has done a study and written a scientific paper can upload it.

A few standards must be met, however. A screening procedure ensures that the paper is indeed a scientific paper and not just some pages from the telephone book. The paper is also checked for its language and excluded if it contains offensive, violent or dangerous wording. If all this is above board, the paper is uploaded.

Preprint servers are not a completely new phenomenon. The first such platform, arXiv, went online back in 1991. However, up to now, they have rarely played such a major role. More than 3,300 studies on the novel coronavirus have been published on bioRxiv so far.

The infectiologist Oliver Cornely is very skeptical about this development. Cornely is scientific director at the Center for Clinical Studies in Cologne. "I have never published anything as a preprint before and I don't see why I should," he says.

Preprint publications have no relevance to his scientific work, he says. "I attach great importance to the peer review process. I don't want to see it before that," says the physician. "What's the difference between a preprint server and a tweet?"

Faster, more open, less thorough?

With this question, Cornely sums up both the advantages and disadvantages of preprint servers: Like a tweet, research results on the preprint servers are made available to a broad public, which is a curse and a blessing at the same time.

Ulrich Dirnagl, for his part, thinks the preprint servers are good. Dirnagl works and researches at the Charite Hospital in Berlin, focusing largely on strokes. Dirnagl also heads theQUEST Center at the Berlin Institute of Health (BIH). There, he and his colleagues are responsible for the quality management of biomedical research at BIH. As a "researcher on research," his field includes looking at the pros and cons of preprint studies.

Dirnagl not only uploads his own publications to bioRxiv and medRxiv, but also uses the servers himself to learn about what his fellow researchers have been finding out. He feels that the speed with which data is being made available to the research community on such servers is a great advantage. "As soon as the study is released, a very intensive discussion of the work within the community begins," said Dirnagl.

Dirnagl finds these discussions about publications in his special field very constructive and helpful, because they give study authors the chance to take on criticism before the lengthy peer review process begins. This means that before being submitted to specialist journals, studies may have already been optimized by several experts.

Unwanted results also visible

Dirnagl says that because the threshold for publication is much lower, preprint servers have another advantage in that "unwanted or unexpected results, so-called zero or negative results, are also more likely to be published."  The German Research Foundation (DGF) has also issued a statement in which it recognizes the advantages of preprint studies.

Most political decisions about how to deal with SARS-CoV-2 are being based primarily on preprint studies. The virologist Christian Drosten also regularly talks about the latest preprint publications in his podcast. He uses his expertise to assess and classify the results — a kind of downstream peer review.

In its statement, the DGF does point out one particular danger associated with preprint servers. "It is problematic when attempts are made from outside the scientific community — by journalists, for example —  to evaluate results whose preliminary nature is not clearly apparent," the DGF says. For this reason, preprint studies on the servers are all given a kind of stamp to denote their provisional character.

Ulrich Dirnagl also recognizes this problem. "Studies that seem very scientific are coming to public attention even though they are complete nonsense," he says.

Peer review in the fast lane

No wonder Oliver Cornely insists on the peer review process. But he says that when things are as urgent as they are at the moment in the midst of the coronavirus crisis, with quick action required, professional journals are prepared to accelerate the entire publication process. 

"One complication that we observed in our COVID-19 patients was a fungal infection of the lungs," said Cornely. "So we wrote a paper very quickly but with a proper peer review process, got questions back, revised the article and finally published it." It took just four or five days, he recalls. Cornely says that many specialist journals have created such fast-lane processes for studies whose publication is particularly urgent.

No guarantee for good research

Ulrich Dirnagl warns, however, against blindly trusting the peer review process. He says that the competition among scientists can prompt many reviewers to give a paper a poor evaluation because they would like to be doing the study themselves.

What is more, according to Dirnagl, "the peer review process practically never prevents real falsifications. The major science scandals were not uncovered by the reviewers, but by people who looked at the articles and said, 'Something is wrong.'"

There are even researchers who would like to abolish peer reviews altogether. Although Dirnagl does not think much of this idea, he does see a need for reform. He would prefer a combination of preprint and peer review.

So-called registered reports can also help to avoid gross mistakes,  says the scientist. Here, the concept of the study — its hypothesis and methods — are assessed before the researchers get down to work. "It's much more sensible to get an opinion from the experts on what you are planning," says Dirnagl.

For COVID-19 research, however, this upstream assessment plays only a minor role: There's probably no time for it right now.