(Lack of) Replication of ESP experiments
A Skeptical Trump card?
(Definition of psi here; definition of ESP here, and definition of precognition here).
Last week I had the pleasure of interviewing Professor Chris French, who is one of the UK’s leading skeptics. (The interview will be out in February for season 2). During the interview, Chris expressed frustration at his inability to replicate significant effects in parapsychology experiments. He also pointed to serious replication problems with one particular kind of parapsychology experiment: unconscious precognition.
Some background:
In 2011, the psychologist Daryl Bem published a paper in the mainstream Journal of Personality and Social Psychology (Bem 2011). Bem’s paper summarised 9 experiments that seemed to show small but significant precognitive effects across several different experimental protocols. This claim sparked a furious controversy in psychology, which I won’t detail here. Anyone who’s interested in the details can read Chris Roe’s fairly comprehensive summary (Roe, 2024).
One key question was whether this effect had been successfully replicated. Some labs seemed to get results, while others didn’t. One standard way of resolving this sort of dispute is called meta-analysis. This is where many studies are statistically pooled to see whether an overall significant effect can be found. In 2016, Bem and team published a meta-analysis of 90 experiments which seemed to show that there was indeed a significant effect (Bem et al., 2016).
However, in 2023, Chris pointed to a large-scale failure to replicate the same kind of precognitive effect in the Transparent Psi Project (French, 2023). (Those interested in the project can read the details here; Duggan, 2023). French concluded from this project that “this massive and rigorous multi-lab replication attempt demonstrates as conclusively as humanly possible that the original [precognition] effect is not real”. (French, 2023). For French this bolsters an earlier, failed replication effort by Stuart Ritchie, Richard Wiseman and he. They also apparently had considerable difficulty getting this failed replication published (French, 2012).
This was by no means the first time that a lack of replication has been cited as a major reason to disbelieve the claims of parapsychologists. In 1999, Richard Wiseman and Julie Milton published an updated meta-analysis to another experimental psi paradigm, the ganzfeld, that showed only chance effects. Their conclusion was that “The ganzfeld technique does not at present offer a replicable method for producing ESP in the laboratory”. (Milton & Wiseman, 1999). I remember this paper causing quite a lot of upset at the time, because the ganzfeld was seen as the flagship experiment of parapsychology.
In his 2024 book, the Science of Weird S**t, Chris further suggests that the original, positive results reported by Bem were likely either flukes or the result of what are known as Questionable Research Practises (French, 2024). He links such problems with what has become known as the Replication Crisis in the social and medical sciences. This is where even commonly-accepted statistical effects have proved surprisingly hard to replicate. This is forcing a deep-reexamination of many standard practises in experimental science.
Partial Arguments?
I’ve always taken the informed critiques of skeptics seriously. Very often these critiques can help tighten up problems with experiments and help move experimental parapsychology forward. I also have a great deal of sympathy for those researchers who fail to replicate psi results in the laboratory. This has sometimes been enough to convert some former psi advocates into skeptics. The case of Susan Blackmore is perhaps the best known. Blackmore became a skeptic after repeated failures to get positive results (Blackmore, 1996).
However, outright skepticism is not the only possible perspective on the psi debates. I’d point to the discussion between Chris French and psi advocate Chris Roe at the British Psychological Society for an overview of two contrasting views (Roe & French, 2024). This debate brings out not only significant differences in opinion but also very different takes on what the data means. Anyone who wants to make a definitive judgment about psi needs to make the effort to understand these differing perspectives.
Any reliance on the narrative promoted by just one ‘side’ in these debates will generally give one a skewed picture of the overall status of the evidence. Robert McLuhan claimed in his review of Chris French’s book that the narrative offered was “partial and incomplete” (McLuhan, 2024, p. 189). This, I think, is a fair critique.
And yet the narrative of any side in this sort of dispute will also be partial and incomplete. Partiality goes both ways. When French ran a course on anomalistic psychology, he used to claim that it would be possible to convince his audience of the truth of the psi hypothesis in about a day (French, 2024). He meant that if one downplays or ignores contrary arguments, which includes arguments about fallibilities in human cognition and perception as well as replication problems, then acceptance of psi might seem inevitable.
The ‘team believer’ vs ‘team skeptics’ frame so beloved of shows like Uncanny won’t help us very much in understanding the division here. A 2024 study of the psychology of the cognitive styles of psi researchers versus skeptics found that the researchers actually had more in common with the skeptics than they did with lay advocates of the paranormal. In other words, both psi researchers and skeptics were aware of the importance of critical thinking and this included being alert to possible errors and problems in experiments that seemed to show psi (Pehlivanova, Weiler & Greyson, 2024).
So why take psi claims seriously?
Despite replication problems, I still think that the evidence points, perhaps slightly, in the direction of some form of psi being real. This position is undoubtedly influenced by my broader worldview. I see the world as being fundamentally interconnected at a basic level. I can see strong evidence for this in physics, biology, and a dozen other sciences. This basic interconnection is, I think, often subtle and still poorly understood. For me, psi phenomena are likely not some special ‘superpower’ or ‘force’, but instead just one expression of that pervasive, often unexpected interconnectedness.
And as Chris French himself suggested, it’s possible to build a reasonable, evidence-based case for psi. I’d point here to two pieces of evidence that incline me in the direction of psi being real. The first is a series of meta-analyses by Patrizio Tressoldi which I list below in the references, with links. These update analyses of the Ganzfeld, Remote Viewing and forced-choice experimental designs. The researchers discovered small but robust effects in all three of these databases (Tressoldi & Storm, 2024A, Storm & Tressoldi, 2023; Tressoldi & Katz, 2023). A further study showed no evidence of a decline in effect size in a survey of five types of Extrasensory Research over a number of decades (Tressoldi & Storm, 2024B). These new surveys include a consideration of the influence of possible Questionable Research Practises.
Secondly, psi ‘hits’ can sometimes be very accurate indeed. One example is given in Russell Targ’s book The Reality of ESP (Targ, 2012). Targ was a key researcher involved with the remote viewing ‘psychic spy’ program in the US. On one occasion, two of Targ’s psychics were able under controlled conditions accurately to describe a secret underground intelligence facility a quarter of a mile distant. This included reporting accurate details of the physical layout and even secret code words from the file drawers lining one of the rooms (Targ, 2012, pp. 23—5, 49—50).
Citing a single incident like this is of course vulnerable to critique. I could, for example, be accused of cherry-picking data or subjective validation, which was a problem for some early Remote Viewing studies (discussed in French, 2024). And yes, one incident doesn’t prove anything. On the other hand, if psi did exist we’d surely expect to see this kind of accuracy at least occasionally. I myself have witnessed a strikingly accurate, comparable ‘hit’ during a controlled ganzfeld experiment. Overall, the persistent phenomenology of such hits, along with the meta-analyses, seems enough to suggest to me that something is going on here beyond chance and experimental error.
So what about replication? Like Chris, I’d like to be able to walk into a lab and replicate positive psi results at will. Like I said, I fully understand the frustration when this does not happen. One way forward might be to pay closer attention to the phenomenology of psi. Understanding ‘experimenter effects’, where some experimenters get results and others don’t, also seems crucial. Thirdly, I take seriously the philosopher Stephen Braude’s claim that positive results might be inherently difficult to replicate, given commonly assumed properties of psi (Braude, 2002).
Finally:
I personally think it likely that psi, if it exists, is inherently subtle and only prone to manifest under specific conditions. One necessary condition may be some kind of altered state. There are good reasons to think this. Many spontaneous psi experiences are reported in the ‘twilight zone’ between waking and sleep or even during dreaming. Psi experiences during waking consciousness do occur but seem rarer (Rhine, 1981).
One reason might be that any subtle psi effect tends to get ‘washed out’ in overstimulated nervous systems. Given that most of us likely have overstimulated nervous systems from excessive smartphone use, etc. these days, this could be a real problem. Patrizio Tressoldi has suggested that one way forward might be to recruit skilled participants and request them to perform tasks in a modified state of consciousness (Tressoldi, 2022).
In sum: I think that replication problems need to be taken seriously by psi advocates as well as psi counter-advocates. I also think that informed skeptics have done parapsychologists a favour by drawing attention to this problem.
References
Bem, D.J. (2011a). Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect. Journal of Personality and Social Psychology 100, 407-25.
Bem, D.J., Tressoldi, P.E., Rabeyron, T., & Duggan, M. (2016). Feeling the Future: A Meta-Analysis of 90 Experiments on the Anomalous Anticipation of Random Future Events. F1000Research. https://f1000research.com/articles/4-1188/v2
Blackmore, S.J. (1996). In Search of the Light: Adventures of a parapsychologist. Prometheus Books.
Braude, S. (2002). ESP and Psychokinesis: A Philosophical Examination. Brown Walker Press
Duggan, M. (2023). Transparent Psi Project.’ Psi Encyclopedia. London: The Society for Psychical Research. https://psi-encyclopedia.spr.ac.uk/articles/transparent-psi-project
French, C. (2024). The science of weird s**t: Why our minds conjure the paranormal. MIT Press.
French, C. (2023). The Transparent Psi Project: The results are in, so where are the headlines? The Skeptic, 16/3/23. https://www.skeptic.org.uk/2023/03/the-transparent-psi-project-the-results-are-in-so-where-are-all-the-headlines/
French, C. (2012 15 March). Precognition studies and the curse of the failed replications. The Guardian. https://www.theguardian.com/science/2012/mar/15/precognition-studies-curse-failed-replications
McLuhan, R. (2024). Review of French, 2024. J SPR 88:3, pp. 187—191.
Milton, J., & Wiseman, R. (1999). Does psi exist? Lack of replication of an anomalous process of information transfer. Psychological Bulletin 125/4, 387-91.
Milton, J., & Wiseman, R. (1999). Does psi exist? Lack of replication of an anomalous process of information transfer. Psychological Bulletin 125/4, 387-91.
Pehlivanova, M. Weiler, M., & Greyson, B. (2024). Cognitive styles and psi: psi researchers are more similar to skeptics than to lay believers. Frontiers in Psychology, 15. https://doi.org/10.3389/fpsyg.2024.1398121
Rhine, L. (1981). The Invisible Picture: A Study of Psychic Experiences. McFarland & Co Inc
Roe, C. (2022). ‘Feeling the Future (Precognition Experiments)’. Psi Encyclopedia. London: The Society for Psychical Research. https://psi-encyclopedia.spr.ac.uk/articles/feeling-future-precognition-experiments
Storm, L. & Tressoldi, P. (2023). Assessing 36 Years of the Forced Choice Design in Extra Sensory Perception Research: A Meta-Analysis, 1987 to 2022. Journal of Scientific Exploration, 37,3, 517-535. http://www.patriziotressoldi.it/cmssimpled/uploads/includes/MAForcedChoice_23.pdf
Targ, R. (20120. The reality of ESP: A physicist’s proof of psychic abilities. Quest Books.
Tressoldi, P. (2022). Further null evidence of extra-sensory-perception by using forced-choice experiments with unselected participants in an ordinary state of consciousness. https://doi.org/10.31234/osf.io/pvbd9
Tressoldi PE and Storm L. (2024A). Stage 2 Registered Report: Anomalous perception in a Ganzfeld condition - A meta-analysis of more than 40 years of investigation [version 4; peer review: 2 approved, 1 not approved]. F1000Research 2024, 10:234 https://doi.org/10.12688/f1000research.51746.4
Tressoldi, P., & Storm, L. (2024B). The Myth of the Decline Effect in Psi Research: The Empirical Evidence. Journal of Scientific Exploration, 38(3), 461-465; https://doi.org/10.31275/20243313
Tressoldi, P. & Katz, L. (2023). Remote Viewing: A 1974-2022 Systematic Review and Meta-Analysis. Journal of Scientific Exploration, 37(3), 467-489. http://www.patriziotressoldi.it/cmssimpled/uploads/includes/MARV_23.pdf