How One Doctor Is Waging War On Bulls**t Science

It's about time.

How One Doctor Is Waging War On Bullshit ScienceWe all know science is a fantastic tool for explaining our world. But as medical doctor-turned-editor Ivan Oransky knows all too well, it’s not perfect.

Oransky, global editorial director of the news site MedPage Today and co-founder of the website Retraction Watch, has made a career of spotlighting bad research — scientific studies so flawed that they aren’t worth the paper they’re printed on or the pixels they’re displayed with.

Oransky estimates that of the 2-3 million scientific papers published worldwide each year, 500-600 wind up being retracted.

That’s not a huge number, but it’s troubling nonetheless, especially since it seems likely that many errant or fraudulent studies are never identified as such.

What’s going on here? Who’s to blame for this shoddy research — and how do they get away with it? More to the point, how can those of us eager to learn the latest scientific findings avoid being misled?

How One Doctor Is Waging War On Bulls Science
Dr. Ivan Oransky

What explains all this junk science?

It’s important to distinguish what people commonly refer to as “junk science” from fraud and retractions. There are fraudulent studies, which are a tiny rarity.

There are studies that don’t hold up when they’re repeated, which can be more than half of the research in some fields, but have very little or nothing to do with fraud.

There are the hundreds of thousands of papers published every year in what are known as “predatory journals” that don’t really have any quality mechanism.

And then there are perfectly well-performed studies that were designed poorly, sometimes in a way that guarantees a result someone wants.

About two-thirds of the time, the cause is something that would be considered misconduct: faking the results, making them look better than they are, or good old-fashioned plagiarism. About one in five retractions is due to honest error.

Can you give some examples of research that was recently retracted?

A couple of much-hyped stem cell papers published in 2014 in the leading journal Nature were retracted after other researchers couldn’t reproduce the findings. It turned out that the papers used bad data.

Another case involved a study in the journal Science that claimed to show gay people were better at changing people’s minds about same-sex marriage. Those data were almost certainly faked.

How come the journals that publish the bad studies fail to spot them in time?

Journal peer reviewers fail to catch the mistakes for a variety of reasons. One that editors usually point to is that if someone is hell-bent on faking results, there’s no way for peer reviewers to catch that.

That’s true even if peer reviewers look at the original data, which is a rare event.

See also: This Health Condition May Afflict Half The World By 2050

But it’s also true that scientists are being asked to review more and more papers, and that science is becoming more and more specialized.

That means there are fewer experts in any particular subject who can vet research.

Who exactly is harmed by bad science?

Certainly other scientists can be harmed, because they can spin their wheels pursuing what’s actually a dead end. And consumers who want better information on which to base decisions can also be hurt. We’re all taxpayers, too, and waste in publicly funded research doesn’t help anyone.

You’ve said you tend to be more suspicious of research in certain scientific disciplines. Which disciplines, and why?

I tend to be much more suspicious of fields that don’t retract as often, which may seem counterintuitive.

But with a few exceptions — physics, for example, which doesn’t retract partly because most scientists allow for review of their results in public long before they’re “published,” making retractions much less necessary — if a field has a higher rate of retraction, that means someone is actually paying attention and trying to correct the scientific record.

There are no fields that have a perfect record that would merit zero retractions.

What do you look for when you read a study?

I always look for the authors’ discussion of the limitations of the study. If they don’t mention any, I run. If they do, it helps me put the findings in context. I figure out whether a study was published in a peer-reviewed journal.

That’s not a Good Housekeeping seal of approval, but it at least means someone has looked at it before the press release came out.

I see whether the study was done in humans, or mice. And I see where it was published. If it was trying to make a claim about clinical medicine, but ended up in a basic science journal, I’m suspicious.

I look for what’s known as a power calculation, to see if the study was big enough to show an effect.

I look for side effects, how many people dropped out of the study, how much the drug or other intervention will cost, who funded the work, and whether there are alternatives. And I seek an outside source, someone without a dog in the fight, to put the findings in context.

Many studies are retracted after being touted in the media. Aren’t journalists supposed to know good science from bad?

I wasn’t born knowing how to separate good science from bad. To those who were, congratulations!

If scientists are letting bad science through peer review, which journals have tried to convince us is necessary before we write about anything, doesn’t it stand to reason that some journalists will assume the vetting process has weeded out the junk?

Journalists have to be eternally vigilant.

What can all of us do to avoid being duped by bad science?

I’d suggest taking a look at HealthNewsReview.org and sites like it that critique news coverage, to get a sense of what you might be missing if you don’t read skeptically.

There are a lot of really good science reporters out there, some at major outlets and some at smaller ones.

Take a look at a journalist’s record, and how often he or she includes caveats and context, rather than going for a simplistic approach that obscures the true story.

Is our system of biomedical research essentially broken? If so, what’s the fix?

As Christie Aschwanden concluded in an excellent piece on FiveThirtyEight, science isn’t broken.

But the incentives under which it’s practiced — a.k.a. “publish or perish” — and the scientific publishing system are broken. That’s what needs fixing.

This post has been lightly edited for clarity.

Find us here

Get news from the CSGLOBE in your inbox each weekday morning

The views and opinions expressed in this article are those of the authors/source and do not necessarily reflect the position of CSGLOBE or its staff.

Paid content

The 10 Biggest Dangers Posed By Future Technology

It’s not easy predicting the future of technology. In the fifties it seemed a pretty much foregone conclusion that by 2015 we would all...

If You’re Alive in 30 Years, it’s Likely You Will be Alive in 1000 Years

Life extension is no joke. It's being pursued, avidly, by the chief engineer at Google, Ray Kurzweil, and is increasingly popular, with some scientists...

Did we discover “Alien technology” and reverse-engineer it?

A lot of people have wondered ever since the first reported UFO crash, if we had come into contact with Alien technology. Highly advanced technology...

What's New Today

Georgia House Votes To Allow Citizens To Abolish Police Departments In The State

The Georgia House backed an effort on Friday to dissolve the Glynn County Police Department and any...

Leaked CDC document contradicts Pence claim that U.S. coronavirus cases ‘have stabilized’

Even as Vice President Mike Pence wrote in a Wall Street Journal op-ed published Tuesday that coronavirus...

Five bombshells about Trump from Bolton ‘s book

Excerpts from former national security adviser John Bolton ’s book about his time in the Trump administration...

Don’t Listen to Fox. Here’s What’s Really Going On in Seattle’s Protest Zone.

It seems I live in a city undergoing a “totalitarian takeover” that will lead to “fascist outcomes”...

MOST READ

What Is Agenda 21? Depopulation of 95% of the World By 2030

Most people are unaware that one of the greatest threats to their freedom may be a United Nations program which plans to depopulate 95%...

Don’t Listen to Fox. Here’s What’s Really Going On in Seattle’s Protest Zone.

It seems I live in a city undergoing a “totalitarian takeover” that will lead to “fascist outcomes”...

Five bombshells about Trump from Bolton ‘s book

Excerpts from former national security adviser John Bolton ’s book about his time in the Trump administration...

The 10 Biggest Dangers Posed By Future Technology

It’s not easy predicting the future of technology. In the fifties it seemed a pretty much foregone conclusion that by 2015 we would all...

Chinese scientists genetically modify human embryos

Human embryos are at the center of a debate over the ethics of gene editing. In a world first, Chinese scientists have reported editing the...

Cars Made Out Of & Powered By Hemp Are The Future

Automobile manufacturer Renew has built cars made from carbon-negative hemp material, with the ability to run on hemp ethanol if it ever were to...

Ingredient in Olive Oil Looks Promising in the Fight Against Cancer

Oleocanthal kills cancer cells with their own enzymes A Rutgers nutritional scientist and two cancer biologists at New York City’s Hunter College have found that...

‘Proof of life’: WikiLeaks asks internet for best way to debunk Assange death rumors

Rumors of Julian Assange’s death are still circulating after his internet connection was abruptly cut more than a week ago. Now WikiLeaks is asking the...

Vladimir Putin Declares War on the Zionist Media

Putin is still stretching his political muscle. He has recently declared that there is an information war going on. He said that for the past...

Cullis-Suzuki: The girl who silenced the world

When Severn Cullis-Suzuki stepped on stage at the plenary session of the Rio Earth Summit in 1992, she knew this was her one opportunity...

The Indian Government Just Eliminated 80% of the Country’s Cash

The Indian government took unprecedented action last week when it eliminated 500 and 1,000 rupee paper notes from its financial system. The change occurred...