I have a headache, so I may be missing something obvious (apart from the fact that I shouldn't be looking at a monitor if I have a headache) but Francis says "Evidence for publication bias in a set of experiments can be found when the observed number of rejections of the null hypothesis exceeds the expected number of rejections" and he uses Bem's precognitive habituation work as an example: 9 replications out of 10 when the effect being described as being small would suggest that not all the data is being presented.
But the Ganzfeld has nothing like that kind of hit rate. Depending on where you look, the percentage of ganzfeld experiments that reject the null (at p<0.05) is between 22-27%.
Just to clarify, 22% comes from my database (38 replications out of 169) while 27% comes from Storm, Tressoldi et al's database (30 replications out of 109 experiments).
I think you're confusing Francis's (then-)definition of publication bias with the test he used to detect it. Francis defined publication bias to include both failure to publish null or disconfirming findings, p-hacking, and exercise of researcher degrees of freedom. What I was saying is that, so defined, publication bias is a plausible explanation for non-null ganzfeld results.
If the test that Francis used (which we now call the test for excess success [TES]), when applied to a set of experiments, results in a small p-value, then the set of experiments almost certainly is biased. But the converse is not true: if a set of experiments is biased, the TES p-value will often not be small. In fact, as Francis's extensive simulations have shown, the TES has very low power to detect bias. So the fact that a set of ganzfeld studies wound pass the TES, does not suggest that the set of experiments is unbiased.
Last edited: