One of the most ‘interesting’ academic papers of recent times is Daryl Bem’s paper in the Journal of Personality and Social Psychology (a reputable journal) on psi. (Yes, psychic phenomenon, he was interested in testing psychic powers). He purported to show that people had the ability to ‘feel’ the future and so predict it better than chance. This violated our standard notion of time going forward.
A Major Challenge For Experimental Research
This paper set a major challenge for experimental research. Bem, a well-respected figure, demonstrated support for something that most people think is nonsense. Is the problem with our closed-minded views? Is the problem with lab experiments? Can you “prove” anything you want to with a lab experiment?
There are a number of potential hits against Bem’s work from a methodological perspective. I would, however, say that Bem’s paper is certainly not obviously worse on this front than many other papers. Clearly, he got put under more of a microscope. That said, I’m not aware that he did anything worse than many others methodologically. There are certainly tricks experimenters use that cast doubt on a lot of papers. Bem didn’t invent the tricks though. Since then experimenters have moved (a little, it could be more) towards better practice so in many ways the paper did everyone a great favor.
Methodological Arguments Seem Secondary
That said, the methodological arguments seem to be secondary to me. Being honest I don’t think he could have done anything in the lab to substantially move my disbelief in psi. Rather than call myself closed-minded I’d prefer to refer to my strong priors. I had a very strong prior that psi doesn’t exist. Maybe Bem moved me a tiny, tiny, tiny, tiny, tiny bit towards thinking that maybe there is something there. That said, the move was basically imperceptible. Sadly, I still don’t believe in psi in any measurable way whatsoever.
I would say that the lack of theory is a fundamental problem with this (and a lot of other work). Without a proper theory of how the psychic process might work any empirical findings are not very compelling. Without a strong theory of how we might feel the future experimenter error seems a lot more plausible than psi as an explanation for the findings.
Testing Psychic Powers
I’d take comfort by noting that Bem suggests that many would agree that “‘extraordinary claims require extraordinary evidence'” (Bem, 2011, page 407). The problem with Bem’s paper is that it is very ordinary. I don’t mean that as an insult, just an observation. The paper is ordinary in the sense that it does what people in the field usually did. It is an exclusively lab-based demonstration. It doesn’t draw in work from outside its field except in a rather superficial way . What is more the author doesn’t support the lab work with any other type of evidence.
Psychologists were right to work on improving their methods post-Bem’s paper but I think we all benefit from getting out of our methodological silos. There is life (but probably not psi) beyond the lab.
For more on experiments see here, here and here. For more on nonsense see here.
Read: Daryl J. Bem (2011) “Feeling the future: experimental evidence for anomalous retroactive influences on cognition and affect.” Journal of personality and social psychology 100.3 407-425