Science in Crisis

Science in Crisis-its worse than you think

By James Corbett

You may have heard of “the crisis of science” recently. That there is something massively wrong with the way science is being conducted these days is not a fringe anti-science idea anymore. It’s being discussed in mainstream publications like The Washington Post, The Economist and The Times Higher Education Supplement, and even mainstream science publications like Scientific American, Nature and phys.org. So what is the problem? And how bad is it, really? And what does it mean for an increasingly tech-dependent society that something is rotten in the state of science? Let’s take a look at the problems facing modern science, and what is at the root of it all.

Irreproducibility

Reproducibility is one of the bedrocks of the scientific method. In a nutshell, an experiment is reproducible if independent researchers can run the same experiment and get the same results at a later time and date. It doesn’t take a rocket scientist to understand why this is important. If an experiment is truly revealing some fundamental truth about the world then that experiment should yield the same results under the same conditions, anywhere and any time (all other things being equal). Well, not all things are equal.

The Center for Open Science led a team of 240 researchers who volunteered to try to reproduce the results of 100 psychological experiments. These experiments had all been published in three of the most prestigious psychology journals. The results, published last year in a paper on “Estimating the Reproducibility of Psychological Science,” were abysmal. Of 100 experiments, only 39 of the experimental results could be reproduced. Worse yet for the advocates of scientific infallibility, these results are not confined to the realm of psychology. In 2011, the journal Nature published a paper showing that researchers were only able to reproduce between 20 and 25 per cent of 67 published pre-clinical drug studies. They published another paper the next year with an even worse result: researchers could only reproduce 6 of a total of 53 “landmark” cancer studies. That’s a reproducibility rate of 11%.

These studies alone are persuasive, but the cherry on top came this past May when Nature published the results of a survey of over 1500 scientists, which found that 70% of them had tried and failed to reproduce published experimental results at some point in their careers. The poll covered researchers from a range of scientific disciplines, from physicists and chemists to earth and environmental scientists, to medical researchers and assorted others. These findings come as no surprise to those who have been ringing the alarm bell about irreproducibility for years — like John Ioannidis. He rocked the scientific community with his 2005 paper “Why Most Published Research Findings Are False.” At the time that paper was published there was a sense that a lot of “landmark” study results were being overturned or disproven, but there was little hard data on how widespread the reproducibility problem was. So what is Ioannidis’ reaction to these more recent findings, which back up his thesis? “I wish I had been proven wrong.” So what’s going on here? Why are so many reputable journals publishing “landmark” studies that turn out to be irreproducible? Well, there’s always…

Fraud

We are taught to believe that scientists are a special breed. Motivated only by their curiosity about the universe, these pure-hearted truthseekers would never dream of publishing a false result or deliberately mislead others. Of course that’s total rubbish. As James Evan Pilato and I reported on New World Next Week in June, that same Nature survey that showed that 70% of researchers had tried and failed to reproduce published experimental results also showed that fully 40% of them believed that fraud was “always or often” the cause.

Again, the problem of scientific fraud is nothing new. As the Dictionary of American History relates:

    “Before 1980, only a handful of accusations of scientific fraud were ever proven. In 1981, however, following press reports of a ‘crime wave’ of scientific fraud, the U.S. House of Representatives conducted the first-ever congressional hearings on the subject. These hearings revealed a wide gap in perception of the magnitude of the problem. Prominent scientists testified that fraud in science was rare; that individual allegations were best investigated on an ad hoc, case-by-case basis; and that government intrusion into the evaluation of scientific fraud would place bureaucrats in charge of declaring scientific truth. Prominent journalists and other critics, in contrast, testified that many cases of scientific fraud had likely gone undetected; that the scientific system of self-policing responded inadequately to fraud; and that the government’s substantial financial investment in basic scientific research necessitated undertaking measures to ensure the integrity of the scientific enterprise.”

As it turned out, the critics were right. Congress acted the only way Congress can in such cases: by passing more legislation! They enacted the Health Research Extension Act of 1985 which created a new federal agency to respond to allegations of scientific fraud. But — surprise, surprise! — for some reason, more government didn’t solve the problem. Despite the creation of the government’s Office of Research Integrity, fraud is still rampant in the scientific community. So, if it isn’t pure-hearted curiosity about the world that is motivating researchers to fudge their results, why do they do it?

Publish or Perish

We’ve all heard of “publish or perish” by now. It means that only researchers who have a steady flow of published papers to their name are considered for the plush positions in modern-day academia. This pressure isn’t some abstract or unstated force; it is direct and explicit. Until recently the medical department at London’s Imperial College told researchers that their target was to “publish three papers per annum including one in a prestigious journal with an impact factor of at least five.” Similar guidelines and quotas are enacted in departments throughout academia.

And so, like any quota-based system, people will find a way to cheat their way to the quota. Some attach their names to work they have little to do with. Others publish in pay-to-play journals that will publish anything for a small fee. And others simply fudge their data until they get a result that will grab headlines and earn a spot in a high-profile journal.

It’s easy to see how fraudulent or irreproducible data results from this pressure. The pressure to publish in turn puts pressure on researchers to produce data that will be “new” and “unexpected.” A study finding that drinking 5 cups of coffee a day increases your chance of urinary tract cancer (or decreases your chance of stroke) is infinitely more interesting (and thus publishable) than a study finding mixed results, or no discernible effect. So studies finding a surprising result (or ones that can be manipulated into showing surprising results) will be published and those with negative results will not. This makes it much harder for future scientists to get an accurate assessment of the state of research in any given field, since untold numbers of experiments with negative results never get published, and thus never see the light of day.

This has led to a growing movement calling on journals to publish more studies with negative results. Journals like New Negatives in Plant Science and the Journal of Negative Results are seeking to address this imbalance by publishing only “hypothesis-driven, scientifically sound studies that describe unexpected, controversial, dissenting, and/or null (negative) results.” This is obviously an important step toward correcting some of the bias that has crept in, but it doesn’t address the main problem, which is…

The Elephant in the Room: Funding

Yes, there is an irreproducibility crisis in science. And yes, it is caused by rampant fraud and fudging of results. And yes, the fudging of results is motivated by the “publish or perish” academic environment. But what creates that environment in the first place? The answer isn’t difficult to understand. It’s the same thing that puts pressure on every other aspect of the economy: funding.

In order to get the big research grants, researchers have to prove their worth. In order to prove their worth, they have to publish. In order to publish, they have to come up with new and surprising results. In order to come up with new and surprising results they have to fudge their data. And when they fudge their data, their results are irreproducible. The base of this chain is the money.

Modern laboratories investigating cutting edge questions involve expensive technology and large teams of researchers. The type of labs producing truly breakthrough results in today’s environment are the ones that are well funded. And there are only two ways for scientists to get big grants in our current system: big business or big government. So it should be no surprise that large corporations and politically-motivated government agencies are paying for the types of science that they want. Want to find a way to blow the head off a mosquito with a laser-guided suborbital rocket-launching satellite? The fine folks at Raytheon or Lockheed Martin or Northrop Grumman will be more than happy to write a check! Want to develop the next generation battlefield-deployable fully autonomous combat robot drone? Then I’m sure DARPA can scrape together some grant money for you! Want to find a way to decentralize the power grid so that communities become self-contained and independent?… Well, too bad. There’s no money in that.

The crisis of science is fundamentally a crisis in the way that science is funded. And like everything else, the answer to this problem is decentralization.

Can you imagine a world of peer-to-peer science? Crowdfunded science? Open science? Well some people can. And they are. As with so many other things, we stand on the cusp of what could be a true revolution in the long-prevailing norms of our society, brought about by the vast online experiment that is the internet. Just don’t hold your breath; they’re not getting big corporate or government money to do it.

wakeup-world.com