Stuart Ritchie in Science Fictions looks at the failure of science. To be fair it is hard to argue with a lot of his points. Ritchie rightly points out that there is fraud but probably more common and alarming is bad practice. Scientists don’t mean to do anything too bad but lots of skimming here and massaging there and you end up with a lot of nonsense in the scientific record. This may be especially acute in the social sciences, given a certain amount of natural messiness in the data. Still, problems go well beyond social sciences.
Publish And The Whole System Perishes
All academics I know live with publish or perish. Many are keen to point to that as a source of a failure of science. It is hard to argue that pressure to publish doesn’t give incentives to fraud and corner-cutting. To be fair a potential alternative — where society just pays academics to chill and think but not stress about publishing their work — has some fairly obvious problems too.
Ritchie notes problems are caused because scholars need to find results to publish. Consider that a faculty adviser has incentives to have their Ph.D. students find a result that can be published, rather than conduct a truly impartial test. The faculty adviser subtly, even unconsciously, pressures the Ph.D. student. Thus bad practice seeps through generations of scholars.
…well-intentioned individual scientists being systematically encouraged — some world argue, forced — to accept compromises that ultimately render their work unscientific.
Ritchie, 2020, page 142
The perverse incentives towards bad practices like those below are manifold.
- Salami slice to milk a single idea for many papers
- Throw out data that ‘seems wrong’
- Create a hypothesis to test after you know the results of the test. (It really is amazing how many hypotheses seem to work in our field. I guess that we are all brilliant at predicting what will work).
Faking Reviews And The Failure Of Science
Ritchie discusses how people have deliberately defrauded the academic system. One of my favorite examples was providing fake suggested reviewers for your own paper. Editors often ask those submitting articles for ideas on reviewers to send papers to. This makes some sense — the writer submitting the article presumably knows the literature well and knows who is an expert on the specific area. That said, this is clearly a problematic practice by itself. People can suggest their friends rather than objective reviewers. Worse than friendly reviewers one fraudster was providing fake email addresses and reviewing his own work. This was suspicious — the fake reviewers got back to the editor in a reasonable amount of time. Real academics are always late. (It is very frustrating to work with academics, many of whom are, let’s be honest, comically deadline averse). The fraud was detected because the fake reviewers were just too well-behaved and professional to be genuine academics.
Ph.D. Students Bringing In Money
One of the things I did find puzzling was Ritchie’s comment that “Ph.D. and other students also bring in vast amounts of money” (Ritchie, 2020, page 180). I’m buying that other students (e.g., MBAs) bring in money but Ph.D. students? Most Ph.D. students I know — certainly me when I was one — are a drain on the resources of their schools. They can bring prestige etc… and so can be good investments. Still, if you are expecting Ph.D. students to bring in the cash, at least in marketing academia (I accept disciplines differ), you’ll be sorely disappointed.
Breaking Too Much New Ground: The Failure of Science
Ritchie he has given more thought to suggestions for improvement than most critics. I’m not sure any will solve the problem but he has some ideas worth trying. E..g., making data open, which is becoming commonplace in many fields. I can tell you what happened when I politely asked for data to replicate papers using Tobin’s q. (See here for the work I did). The positive spin is that I wasn’t burdened with having to do any additional analysis to replicate the past work as absolutely no data was forthcoming.
One thing I agree it would be good to get away from is the idea that papers all have to break new ground. Often surprising things are surprising because they are wrong. Even when the new findings aren’t wrong, at some point someone has to do more than break new ground. We need to build on past findings.
To paraphrase the biologist Ottoline Leyser, the point of breaking ground is to begin to build something; if all you do is ground-breaking, you end up with a lot of holes in the ground but no buildings.
Ritchie, 2020, page 204
Criticize The Failure Of Science But Compare To The Alternatives
Ultimately Ritchie comes (somewhat) to praise science, not to bury it. He is right to say science must keep its own house in order to earn the trust it (mostly) still gets from the public. Science might have flaws but it beats the alternative of everyone just speaking unverifiable nonsense. Without the skepticism encouraged by scientific thinking, we’d all just be using Hofstede’s cultural dimensions in our analysis.
For more on similar problems see here and for more on academia see here, here, and here.
Read: Stuart Ritchie (2020) Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth, Macmillan