A Breakdown on Preprints With Eric Topol, MD

[MUSIC PLAYING] Welcome, everyone. I'm Dr. John Whyte, The chief medical officer at WebMD, and you're watching Coronavirus in Context. One of the things that we've seen throughout the pandemic is this rapidly emerging science. And the traditional model is, let's wait until things get published, and then we'll talk more broadly about it. But we don't have the time to do that during literally a pandemic that impacts the world. So what's the role of preprints? Joining me to help explain the pros and cons is my good friend, the editor-in-chief of Medscape, Dr. Eric Topol. Eric, it's great to see you as always.

Great to be with you, John.

One thing we've talked about is a success story of the COVID pandemic has been the speed of innovation. And part of innovation in terms of pharmaceuticals, in terms of diagnostic tests is the ability to peer review, to have an outside group of experts look at it and assess it for its validity or its objectivity.

The pandemic, as I referenced in my remarks, hasn't allowed that. So we never really talked about preprints years ago. And now, if you remember early on in the pandemic, everyone was talking about it. So what's the role of preprint? And maybe we take a step back and help us define, what do we mean by that?

Right. Well, this is really important stuff to get into, John. And I spend a fair amount of my day now every day looking at preprints. The real story here is that they've been going for about 10 years in one platform or another. The one that is mostly affecting us is bio archives and med archives, which is handled by Cold Spring Press, Cold Spring Harbor.

And they are basically manuscripts that are put on that server after they've been screened. So they're not detailed peer-reviewed. But for example, they check if they have had IRB approval, and they basically fulfill good conduct of research, especially when it's clinical, but also lab-based research. And they have not yet gone out to peer review by journals. Although right now, many journals want people who are authors to put it on the preprint server first, because it's basically like the peer review. Because immediately after a good one's posted, Twitter is all over it.

So in a way, it's a whole different form of peer review, and it's really amazing. Like just yesterday I have to say, I kind of got blown away. Here Omicron, as you know, was just reported Thanksgiving, November, end of November. And a paper from Israel was reported with follow up through January 27 on a fourth dose on February 1, OK? So on millions of people, all right? So we would never see that data normally if it only went through peer review. It could take months. Months or maybe--

The argument could be, why don't we speed up peer review? Why don't we make it quicker and easier instead of saying preprint? And I have to ask you, is preprint the right term? Because for those that aren't involved in research like you, preprint, I'm going to be honest, sounds to some people, it's going to be published, right? And this is an early look. That's not the case.

Well, that's true. There are many preprints that never see the light of day. Quite a few are published. In fact, many in leading journals. It really is depending on who's writing the preprint. If it's coming from reputable people, they don't want to put something out there on the web. I mean, their name is on the line, right? So if it's something that's flawed, it's going to get picked up by the masses. I mean, there's so many people that look at every preprint on these different servers.

So I think you have to be very careful about who are the authors, where is it coming from. But your point is essential, John because what you said, well, what about just getting peer review faster? The problem is, when we do peer review, it's two or three people who sometimes don't even really go through the paper carefully. And they take a long time.

Even with the pressure of the pandemic, trying to get it down to a week or two, by the time you get three reviewers in or two reviewers in, and they're late and you have to just keep, the editors have to keep pinging them, it's all done on a voluntary basis. So the real thing is, peer review as we know it is that really the gold or platinum standard? Especially since it takes quite a while. So what you're comparing with are two different ways to get-- you can get a rapid mass review by experts and non-experts through a post on a preprint server.

It's like rapid and PCR in a way. Antigen test versus PCR. But still, people could argue and say change the peer review process. Pay academicians or whoever to review it. Hire more people. Like this is serious research that we really need to determine. But what about the fact where there has been this attack on science throughout the pandemic, people don't like when recommendations change because our knowledge of the science is evolving. And I pointed out, you and I have talked throughout these 24 plus months. Early on, there were a couple preprints that were wrong in terms of--

There's many

--what they thought about treatment, what they thought about infectiousness, et cetera. And people were starting to make decisions on that. Because the media doesn't really make that distinction. That it's preprint.

You're spot on on that.

How do we use preprints in the broader community because if it's not accurate because it's early it's still a little earlier on, that's a challenge. But we also don't want to miss opportunities and it's something that we need to address in terms of public health. How do we put it all together while recognizing sometimes it doesn't meet the rigor that we need?

Right. So the treatment side of the pandemic has been one of the most important abuses and misguided aspects. So hydroxychloroquine was one of the early preprints from France that was completely off base, and it set off this hydroxychloroquine. There's been the same for convalescent plasma, which I reacted to when I attacked the FDA commissioner and the administration September of 2020.

Basically, the preprint has much more contamination than you'll see through high quality peer-reviewed journals, right? And you have to ferret out what is either a likely suspect for contamination, wrong, versus the really high quality ones.

I mean, the body of literature is enormous. It keeps doubling it seems like every few months. How does someone keep up with it? Because we can say, all right, we have all these journals. And there's more and more journals evolving. We have increased frequency. Now we have preprint to the degree that we never had before. How do you realistically keep up with it. I mean, there's only so many hours in the day. One could be reading preprints all day. How do you balance that? How do you how do you utilize preprints?

Yeah. I mean, I basically try to stick with top journals that I know well. And then I go through the three main preprint servers, at least for the pandemic. And I can pretty well look at 10 preprints, let's say, of their titles and authors, and know I'm only going interest to zoom in on one because I know these people or I know their work is impeccable. It's the real deal kind of thing. And I won't waste time looking at ones that I think I don't know the people or the topic is not particularly germane or whatever. That's how I do it.

Most people don't have your expertise. So do we need to create some type of curation almost like an Dr. Eric Topol AI curator to put those things explicit?

I would love to see-- your idea is very good, John-- I would love to see a panel reviewer of each preprint as they come out, like consumer reports and just give it the yellow or the red circle stuff. But we don't have that, and who are we going to get to do that? So you're right, it would help because I think the average person who doesn't know the players, the institutions, the whole landscape.

Like yesterday, there was a really interesting preprint about five people from University of Maryland who died. And in their brain, in their cortex, they had the virus in the cells. And they also had an inflammation activation that looked like Alzheimer's. And I read this, and oh, man, these are really good people. One of the leading Coronavirus virologists was part of this to make this determination.

As you know, it's been very controversial whether the virus gets into brain cells or just induces inflammation. So I shared that because I know it was a good study, and it's of course, a concern. And I shared that it's controversial. And if this is the real deal, we don't know how frequently it happens. And there were people that were upset. There were a few people that said, oh, you shouldn't have shared that. Well, it's actually important. What are we supposed to do, suppress it?

And it's a pretty how should the media use it? Because often then, those caveats that you described aren't mentioned.

Right. Right.

And then people misinterpret the information. So we want to be egalitarian. We want to be transparent. And in some ways, we want to have open access, right? But then how do we make sure that the caveats for preprint aren't misinterpreted?

Yeah. No, I think this is important. That is, you see now in the media this has not been peer reviewed. Yesterday, there were three papers published in Nature, which I regard as one of the top entered journals in the world in bio-medicine. And they were three papers that characterize Omicron's properties. And obviously, this is all done in a matter of weeks and these are extraordinary studies from Cambridge and Japan and whatnot, Hong Kong.

Now, I read those preprints, and I don't see a word different in the preprints than in Nature, all right? So when I read a preprint of that quality, to me, it doesn't matter. And if I was the media, I wouldn't even make it a distinction because as far as I'm concerned, it's the real deal. Obviously, we still want peer review, we want it quick. We also want the journals themselves to get stuff out there. And now you see the journals have moved at velocity that I don't think any of them thought they were capable of.

That is true.

So it's good. And the problem is finding reviewers. The reviewers are getting burn out. There's more publications than ever before. So also, you also mention about all these other journals out there that are peer-reviewed that publish anything, predatory journals, right? That basically they just charge money, and then they're doing it for profit. So we have junk peer review too.

How do people distinguish it? We've talked about curation, other types of technologies. Should the public not go to preprint servers?

What I like about the preprint is, you'll see anything that's important. It'll get thoroughly vetted.

It's like crowdsourcing. It's like crowdsourcing peer review in a way.

Yeah. Basically, Twitter is the disinfectant, the transparency. It's hitting the light of day so you get the expert. You actually get more peer review by many more people. They tend to be savage, unfortunately. That's the one thing I'd say. But you know what, it's better than no peer review.

And then we see more of the data, correct? We're going to see more of the actual data as opposed to some journals where there's more tabulations and less of the actual primary data.

Yeah.

Not always, but sometimes.

Yeah, I agree with that. I mean, I think that we have-- there are some people, leading people in academics who actually think that Twitter experts who get on and put in their concerns or are their positive feedback that may be the best peer review system.

So what does the publication of research look like two years from now?

It's really a great question. I think what we have learned is, we're intolerant of what used to be the time to get important things published. Journals have shown us that they don't have to do it that takes months. It's been compressed just like vaccines were compressed to be done in days, and sometimes even in hours. So I think everything is changing now. We're moving to a more accelerated expectation.

Now, how do we do that while preserving the highest quality with or without mass input via Twitter or other social media? Maybe we need a different media that's not like Twitter that's pure vetting preprints. And you can't be on it unless you get credentialed, and you basically-- that might be the solution because this whole somewhat--

Should third party score it in a way, almost?

Yeah. I mean, right now, Twitter obviously is public, and it's not really necessarily only for scientists. And yeah, anybody from the public can see what these people are saying. Some may understand it, some may not. But the problem is-- and when you have papers on things like ivermectin or hydroxychloroquine, you'd like the public to see what the flaws are.

But on the other hand, Twitter was never intended for that. We don't really have the ideal platform for mass, quick vetting. And something to think about in the future that would also be shareable to the public, but public people who are not experts or not credentialed wouldn't be doing the critiques necessarily.

Dr. Topol, I want to thank you again for sharing your insights, helping us dissect exactly how do we use preprints, and what's the future because they are here to stay. And we're only going to see more use of this type of publishing. So thank you, Dr. Topol.

Oh, thank you, Dr. Whyte John. Always a pleasure to be with you.