Ganzfeld Experiments: Suggestions please.

#1
My first introduction to parapsychology came at least 1 and a half year ago with the Ganzfeld Experiments. Since then, I've been studying both skeptical and pro-psi proponents, as far as my money, time and resources allow me (I don't have enough cash to buy expensive papers/books, and my language skills are limited to spanish and english). In the time, I've meet four different potential objections to it that I've been unable to find any reply from the pro-psi proponents, and one seems to be so obscure that I've actually never found anything at all in the Internet about it. Here I'll put them, for two reasons: (1) Further discussion and positive feedback, and (2) ask you, ¿what should I do next? I've find myself in a dead end concerning this issue. ¿Should I move into another parapsychology topic?

So, here are the four potential objections:

(a) Kennedy's paper concerning power size: This one is pretty recent, actually from last year. You can find it for free here (http://jeksite.org/psi/jp13a.pdf). I came to know about this paper by Prescott's blog where he (Kennedy) and Carter engaged a little bit with the topic. I've also seen Arouet use it in a post concerning the Ganzfeld in this forum, but there where no further replies I'm aware of. The central issue in Kennedy's paper is that under-power-sized meta-analysis tend to give incorrect data, specially if they are post-hoc.

(b) Goodfellow's Objection: I've only found this particular objection in a book named "Introduction to Parapsychology" (5th edition) in the chapter concerning theories for psi. Sadly, my Kindle is death so I can't quote however I do recall a poster named "Linda" in the James Randi Foundation making a similar post. Namely, that the assumption that 25% is the expect chance may be wrong because people usually have a bias toward certain numbers, like the first and the last options showed, and the shuffling distribution of the target many not always follow a 25/25/25/25 pattern. Apparently Goodfellow found evidence of this effect (where people choose in a non-random way, like first, first, second, second).

(c) Hardy's Objection: From this, I've only been able to found a small reference in the skepdic, which I'll quote directly (http://www.skepdic.com/psiassumption.html):

"
Studies comparing random strings with random strings, to simulate guessing numbers or cards, have found significant departures from what would be expected theoretically by chance (Alcock 1981: 159). For example, Harvie “selected 50,000 digits from various sources of random numbers and used them to represent “target cards” in an ESP experiment. Instead of having subjects make guesses, a series of 50,000 random numbers were produced by a computer.” He found a hit rate that was significantly less than what would be predicted by chance (Alcock 1981: 158-159).

In the 1930s, Walter Pitkin of Columbia University printed up 200,000 cards, half red and half blue, with 40,000 of each of the five ESP card symbols. The cards were mechanically shuffled and read by a machine. The result was two lists of 100,000 randomly selected symbols. One list would represent chance distribution of the symbols and the other would represent chance guessing of the symbols. However, the actual matches and what would be predicted by accepted odds didn’t match up. The total number was 2% under mathematical expectancy. Runs of 5 matching pairs were 25% under and runs of 7 were 59% greater than mathematical expectancy (Christopher 1970: 27-28). The point is not whether these runs are typical in a real world of real randomness or whether they represent some peculiarity of the shuffling machine or some other quirk. The point is that it is not justified to assume that statistical probability based on true randomness and a very large number of instances applies without further consideration to any finite operation in the real world such as guessing symbols in decks of 25 cards shuffled who knows how or how often, or rolling dice, or trying to affect a random number generator with one's mind. As Alcock put it: “If such significant variation can be produced by comparing random strings with random strings, then the assumption that any significant variation from chance is due to psi seems untenable (Alcock 1981: 158-159).”"
d) Ioannidis/Ersby Objection: Ioannidis famous paper from Nature of why so many papers are false seems to give a series of criteria to discern the probability of a study or series of studies of having erroneous data, and I've read some quite convinging cases given as to why Ganzfeld may fit the bill to be labelled as the effect caused by different biases and errors leaking in the studies.

Ersby makes a similar objection, which can be read at the end of his introduction to his work on the Ganzfeld in skeptic's report page. His objection is about his personal recollection of over 7000 Ganzfeld studies, and how, when put in line, they don't show a funnel graph, which is an indicator of a genuine effect. I've read that the file-drawer objection is mislabeled ( Randi has make this counter-objection), but I've also read that there is some controversy as to how exactly detect the file-drawer effect, and that some analysis may be give over-inflated results. Ersby objection can be found at the bottom, here (http://www.skepticreport.com/sr/?p=316).

So, those are the objections that I've found the strongest. ¿Any opinion, suggestions? Thanks in advance.
 
#2
Well, since I'm mentioned by name, I may as well respond. And I'm quite honoured to find my name mentioned alongside Ioannidis.

a) Power size.

Kennedy is right, ganzfeld experiments tend to be under-powered. This is more to do with the lack of funding in parapsychology than any failings on the part of the researcher. The obvious answer is to change to a more sensitive measuring system (order ranking or have more decoys along with the target), but the 25% chance expectation has become almost sacrosanct in ganzfeld work. This is probably why the ganzfeld is being superceded by other protocols such as Bem's precognitive habituation and Radin's presentiment experiments.

b) Goodfellow's objection: Judging bias.

I've never of Goodfellow, so I've added the name I'm more familiar with. The idea that judges tend to choose the first target they see is true, but that has nothing to do with what the actual target is.

For example, Ray Hyman noted that in Honorton's PRL data, judges tended to choose the 1st and 4th placed targets more than chance would suggest. But Bierman showed that, in practice, the target was mostly shown third in the judging procedure. In other words, judging bias would have decreased the expected hit rate by chance.

Judging bias is only applicable after the experiment is done. If the results are strongly positive or negative, the data should be checked to see if this artefact had an impact on the results.

c) Hardy's objection: the psi assumption

Well, yes. Coincidences can happen if you look hard enough. So what?

d) Ioannidis/Ersby objection

Not sure what we have in common but, again, thanks for putting our names together.

Ioannidis' paper is relevant to parapsychology, but it is mostly about mainstream peer-reviewed science. A lot of the issues he brings up in his paper are well known in parapsychology and have been known for several decades.

As for my part of this objection: it's true, the ganzfeld experiments don't form a funnel plot as far as I can tell. Radin has a graph of the ganzfeld data in Entangled Minds that does have a typical funnel shape, but he doesn't specify which effect size he uses.
 
Last edited:
#3
http://www.deanradin.com/evidence/Williams2011Ganz.pdf

Revisiting the Ganzfeld ESP Debate:
A Basic Review and Assessment
BRYAN J. WILLIAMS
University of New Mexico, Albuquerque, NM 87102
bwilliams74@hotmail.com
Submitted: 1/3/2011; Accepted 6/1/2011

Abstract—This paper presents a brief review of the debate between parapsychologists
and skeptics regarding the issue of replication in experimental tests
of extrasensory perception (ESP) using a sensory reduction technique known
as ganzfeld. The review is followed by a basic assessment of 59 ganzfeld
ESP studies reported in the period following the publication of a stringent
set of methodological guidelines and recommendations by R. Hyman and C.
Honorton in 1986. The assessment indicates that these 59 studies have a combined
hit rate of approximately 30%, which is signifi cantly above the chance
expected hit rate of 25%.
A comparison of the hit rates across four ganzfeld
meta-analyses, as well as across fi fteen laboratories, seems to further indicate
replication of the ganzfeld ESP effect by a broad group of independent researchers.
http://www.noetic.org/library/publi...ld-psi-performance-within-artistically-gifte/
Ganzfeld Psi Performance Within an Artistically Gifted Population

by Charles Honorton and Marilyn Schlitz, PhD

The Journal of the American Society for Psychical Research, 86(2).

ESP Ganzfeld study with exceptional population, students from Juilliard School in New York City. Twenty undergraduate Juilliard students each served as a receiver in a single telepathy Ganzfeld session. Juilliard students achieved a significant success rate of 50%, double the chance of expectation. Students of music demonstrated the most outstanding performance; six of the eight musicians obtained direct hits or 75%. As a group, the Juilliard students' performance was significantly superior to the PRL general population.
 
#4
With respect to b)...

While there are position preferences, the research suggests that preferences with respect to target content may have a larger effect. For example, in Bem's analysis of Honorton's auto ganzfeld trials, position biases had little effect, but content biases seemed to account for at least part of the effect. And Wackermann found target preferences in his research (when looked for specifically). As he points out, this prevents us from ruling out stacking effects (instead of 'anomalous cognition').

http://deanradin.com/evidence/Bem1994-2.pdf

http://www.anomalistik.de/images/stories/pdf/sdm/ejpv22-1_puetz%20et%20al.pdf

Linda (former JREF poster)
 
#5
Well, since I'm mentioned by name, I may as well respond. And I'm quite honoured to find my name mentioned alongside Ioannidis.
I have many questions to you, so stay tuned :P


a) Power size.

Kennedy is right, ganzfeld experiments tend to be under-powered. This is more to do with the lack of funding in parapsychology than any failings on the part of the researcher. The obvious answer is to change to a more sensitive measuring system (order ranking or have more decoys along with the target), but the 25% chance expectation has become almost sacrosanct in ganzfeld work. This is probably why the ganzfeld is being superceded by other protocols such as Bem's precognitive habituation and Radin's presentiment experiments.
¿Is there a value in the current evidence of Ganzfeld (in your opinion) fi they are down-powered?


b) Goodfellow's objection: Judging bias.

I've never of Goodfellow, so I've added the name I'm more familiar with. The idea that judges tend to choose the first target they see is true, but that has nothing to do with what the actual target is.

For example, Ray Hyman noted that in Honorton's PRL data, judges tended to choose the 1st and 4th placed targets more than chance would suggest. But Bierman showed that, in practice, the target was mostly shown third in the judging procedure. In other words, judging bias would have decreased the expected hit rate by chance.

Judging bias is only applicable after the experiment is done. If the results are strongly positive or negative, the data should be checked to see if this artefact had an impact on the results.
¿How pervasive is this effect in the Ganzfeld? I've check some papers but I can't find the part where they put the place the target received in the experiment. My claim was more a sort of fussion between Goodfellow's claim and Hardy's claim. If the targets don't receive a 25/25/25/25, ¿is it possible that in some cases, some specific targets get to be more in first or fourth positions, and be selected much more, merely by it's position?

I also recall that Honorton, in his PLR trials, seem to mention something about how "watery" targets tend to be chosen more, and Linda's paper on Wackermann (if I'm reading well) seems to indicate that "elementary" targets ( water, air, fire, earth) tend to be much more chosen than regular ones. ¿Can this account for part of the data?

BTW, IIRC in your "Psi in the Ganzfeld" script, you put that Hyman found evidence for the video degradation hypothesis in the PLR trials, but a personal talk with one of the designers of the PLR trials told you that the videos were played uniformly, but only one was shown in the camera. ¿How did all that end?, Or, in other words, ¿is Hyman hypothesis refuted by this?


c) Hardy's objection: the psi assumption

Well, yes. Coincidences can happen if you look hard enough. So what?

I think this may be better seen in conjuction with Goodfellow's hypothesis.


d) Ioannidis/Ersby objection

Not sure what we have in common but, again, thanks for putting our names together.

Ioannidis' paper is relevant to parapsychology, but it is mostly about mainstream peer-reviewed science. A lot of the issues he brings up in his paper are well known in parapsychology and have been known for several decades.
Had they been adressed in the modification the Ganzfeld has received across the decades?


As for my part of this objection: it's true, the ganzfeld experiments don't form a funnel plot as far as I can tell. Radin has a graph of the ganzfeld data in Entangled Minds that does have a typical funnel shape, but he doesn't specify which effect size he uses.
¿Have you asked Radin for such information? would be interesting to know what he thinks about the issue.

Also, a last question that I've had since I started reading your posts: ¿Do you believe in psi? and ¿Do you believe there is good evidence for psi in the scientific literature of parapsychology?
 
#7
With respect to b)...

While there are position preferences, the research suggests that preferences with respect to target content may have a larger effect. For example, in Bem's analysis of Honorton's auto ganzfeld trials, position biases had little effect, but content biases seemed to account for at least part of the effect. And Wackermann found target preferences in his research (when looked for specifically). As he points out, this prevents us from ruling out stacking effects (instead of 'anomalous cognition').

http://deanradin.com/evidence/Bem1994-2.pdf

http://www.anomalistik.de/images/stories/pdf/sdm/ejpv22-1_puetz%20et%20al.pdf

Linda (former JREF poster)
Same as with Ersby, I encourage you to stay, since I have many questions to you also :P

Now to the topic. I can't find Wackermann acknowledgmenet of the judging bias, but on the other hand, the language he engages in is quite complex so I don't fully understand everything he says. ¿Can you point to me where it is? Also, ¿what do you think about Biermans analysis of Honorton data (as put by Ersby)? ¿Do you think it fits your hypothesis that judging bias may account only partially for the hits in the Ganzfeld?

Also, IIRC, you had in the James Randi Foundation forum a discussion with a poster that, IIRC again, was named Elizabeth, who made a program to check if your hypothesis was correct. ¿How that ended, where can I get that program, and how much of the effect of the Ganzfeld do you think can be explained this way?
 
#8
Same as with Ersby, I encourage you to stay, since I have many questions to you also :P

Now to the topic. I can't find Wackermann acknowledgmenet of the judging bias, but on the other hand, the language he engages in is quite complex so I don't fully understand everything he says. ¿Can you point to me where it is?
"Target specific identification rates", page 64.

Also, ¿what do you think about Biermans analysis of Honorton data (as put by Ersby)? ¿Do you think it fits your hypothesis that judging bias may account only partially for the hits in the Ganzfeld?
I'm not familiar with that analysis. Perhaps a reference could be given.

Also, IIRC, you had in the James Randi Foundation forum a discussion with a poster that, IIRC again, was named Elizabeth, who made a program to check if your hypothesis was correct. ¿How that ended, where can I get that program, and how much of the effect of the Ganzfeld do you think can be explained this way?
A poster named Beth made a program, but it did not model my idea.

The best/easiest way to tell how much of an effect it has is to measure the hit rate in the absence of the effect (or rather, alter the design and analysis of the experiment so that these biases would not affect the hit rate).

Linda
 
#9
a) Power size.

Kennedy is right, ganzfeld experiments tend to be under-powered. This is more to do with the lack of funding in parapsychology than any failings on the part of the researcher. The obvious answer is to change to a more sensitive measuring system (order ranking or have more decoys along with the target), but the 25% chance expectation has become almost sacrosanct in ganzfeld work.
If I may ask, what is the reason for considering more decoys/order ranking? I honestly don't see what difference it will make given that the hits and misses will average out closer to the probability of hit as more trials are included, unless of course, you assume there is a randomization error or flaw in the study. You're right that Ganzfeld experiments tend to be under-powered; however, combining them into a meta-analysis will eventually add more power and better accuracy than the individual studies themselves.

Well, yes. Coincidences can happen if you look hard enough. So what?
True, but such coincidences occur based on the Type I Error probability (which is based on the ά of the test), so if we reject the null when p < 0.05 then we should expect, by average, 5% of studies to be statistically significant when they are nothing but statistical flukes.
 
#10
¿How pervasive is this effect in the Ganzfeld? I've check some papers but I can't find the part where they put the place the target received in the experiment. My claim was more a sort of fussion between Goodfellow's claim and Hardy's claim. If the targets don't receive a 25/25/25/25, ¿is it possible that in some cases, some specific targets get to be more in first or fourth positions, and be selected much more, merely by it's position?
Yes. It is important to ensure that the four pictures are equally selected. Any non-random selection, subtle or large, might reveal patterns that could be noticeable consciously or unconsciously, which will eventually inflate the hit rate and skew the results.

I also recall that Honorton, in his PLR trials, seem to mention something about how "watery" targets tend to be chosen more, and Linda's paper on Wackermann (if I'm reading well) seems to indicate that "elementary" targets ( water, air, fire, earth) tend to be much more chosen than regular ones. ¿Can this account for part of the data?
Not if the four pictures had an equal 25% chance of being the target. Otherwise, yes.
 
#11
¿Is there a value in the current evidence of Ganzfeld (in your opinion) fi they are down-powered?
I think the ganzfeld work has value, sure. Not as much as some commentators say, but something interesting is certainly going on.

¿How pervasive is this effect in the Ganzfeld? I've check some papers but I can't find the part where they put the place the target received in the experiment. My claim was more a sort of fussion between Goodfellow's claim and Hardy's claim. If the targets don't receive a 25/25/25/25, ¿is it possible that in some cases, some specific targets get to be more in first or fourth positions, and be selected much more, merely by it's position?
It's hard to tell, since very few papers include this kind of analysis.

I also recall that Honorton, in his PLR trials, seem to mention something about how "watery" targets tend to be chosen more, and Linda's paper on Wackermann (if I'm reading well) seems to indicate that "elementary" targets ( water, air, fire, earth) tend to be much more chosen than regular ones. ¿Can this account for part of the data?
Possibly yes. But equally possibly no.

BTW, IIRC in your "Psi in the Ganzfeld" script, you put that Hyman found evidence for the video degradation hypothesis in the PLR trials, but a personal talk with one of the designers of the PLR trials told you that the videos were played uniformly, but only one was shown in the camera. ¿How did all that end?, Or, in other words, ¿is Hyman hypothesis refuted by this?
The video degradation hypothesis is weakened by the fact that the target and decoys are all played equally during the sending period, but there's still that odd pattern in the data that targets were identified more often if they'd already been used as targets previously.

Had they been adressed in the modification the Ganzfeld has received across the decades?
I took a look at his paper again. These are the six main part of his conclusion:

The smaller the studies conducted in a scientific field, the less likely the research findings are to be true. - - This is still a problem for the Ganzfeld.

The smaller the effect sizes in a scientific field, the less likely the research findings are to be true. - - This is linked to the one above, and effect sizes in the Ganzfeld tend to be quite small.

The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true. - - This isn't a problem. While a lot of researchers investigate other hypotheses in their work, the main measure of success is pretty much always the same.

The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true. - - Again, not a problem. Designs, definitions and outcomes are all well understood in the Ganzfeld work.

The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true. - - Financial interests are not a problem at all. Prejudices may come into play - this is a controversial subject, after all - but there are prejudices on both sides.

The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true. - - This is a problem. Parapsychology is a very small science.

Also, a last question that I've had since I started reading your posts: ¿Do you believe in psi? and ¿Do you believe there is good evidence for psi in the scientific literature of parapsychology?
I believe that something's going on.
 
#13
I'm not familiar with that analysis. Perhaps a reference could be given.
Dick J. Bierman, Richard S. Broughton, and Rick E. Berger, “Notes On Random Target Selection: The PRL Autoganzfeld Target And Target Set Distributions Revisited”, Journal of Parapsychology vol 62, no 4, December 1998, pp 341-348
 
#14
Dick J. Bierman, Richard S. Broughton, and Rick E. Berger, “Notes On Random Target Selection: The PRL Autoganzfeld Target And Target Set Distributions Revisited”, Journal of Parapsychology vol 62, no 4, December 1998, pp 341-348
Thank you.

Linda
 
#15
d) Ioannidis/Ersby Objection: Ioannidis famous paper from Nature of why so many papers are false seems to give a series of criteria to discern the probability of a study or series of studies of having erroneous data, and I've read some quite convinging cases given as to why Ganzfeld may fit the bill to be labelled as the effect caused by different biases and errors leaking in the studies.
I think you are referring to Ioannidis' paper in PLOS medicine (http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124).

With respect to the corollary "The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true", we went through the process of assessing the risk of bias in included studies for some of the parapsychology research on the old forum (http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm). Doing so for the ganzfeld (surprisingly) revealed that there are still issues present which put included studies at a higher risk of bias. My recommendation all along has been to use this information and, going forward, to design, implement, and analyze ganzfeld experiments in a way which puts them at low risk of bias. The handbook I referenced provides an overview of methods which improve reliability and validity.

Linda
 
#16
A poster named Beth made a program, but it did not model my idea.

The best/easiest way to tell how much of an effect it has is to measure the hit rate in the absence of the effect (or rather, alter the design and analysis of the experiment so that these biases would not affect the hit rate).

Linda
I see. However, IIRC, you made a program to test your idea, and you get positive results towards your hypothesis, is that correct? ¿How can I run the program myself for further development on this idea?
 
#17
Yes. It is important to ensure that the four pictures are equally selected. Any non-random selection, subtle or large, might reveal patterns that could be noticeable consciously or unconsciously, which will eventually inflate the hit rate and skew the results.
Has this possible failure been adressed in the Ganzfeld? I recall quite a few Ganzfeld papers where this option is being analyzed.
 
#18
The video degradation hypothesis is weakened by the fact that the target and decoys are all played equally during the sending period, but there's still that odd pattern in the data that targets were identified more often if they'd already been used as targets previously.
What do you think is the best explanation to account for all the data concerning said issue?


I took a look at his paper again. These are the six main part of his conclusion:

The smaller the studies conducted in a scientific field, the less likely the research findings are to be true. - - This is still a problem for the Ganzfeld.

The smaller the effect sizes in a scientific field, the less likely the research findings are to be true. - - This is linked to the one above, and effect sizes in the Ganzfeld tend to be quite small.

The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true. - - This isn't a problem. While a lot of researchers investigate other hypotheses in their work, the main measure of success is pretty much always the same.

The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true. - - Again, not a problem. Designs, definitions and outcomes are all well understood in the Ganzfeld work.

The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true. - - Financial interests are not a problem at all. Prejudices may come into play - this is a controversial subject, after all - but there are prejudices on both sides.

The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true. - - This is a problem. Parapsychology is a very small science.
In a GRADE scale, how much do you think the Ganzfelds should receive, and why?

Possibly yes. But equally possibly no.
Do you think it's possible to discern between those two options with the avaible information?

I believe that something's going on.
Thanks. ¿But do you think it's something paranormal or some quirk in the design of the experiment?
 
#19
I think you are referring to Ioannidis' paper in PLOS medicine (http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124).

With respect to the corollary "The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true", we went through the process of assessing the risk of bias in included studies for some of the parapsychology research on the old forum (http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm). Doing so for the ganzfeld (surprisingly) revealed that there are still issues present which put included studies at a higher risk of bias. My recommendation all along has been to use this information and, going forward, to design, implement, and analyze ganzfeld experiments in a way which puts them at low risk of bias. The handbook I referenced provides an overview of methods which improve reliability and validity.

Linda
Where can I find that thread? BTW, thanks for the Cochrane link, though I must admit I don't understand everything in it.
 
#20
To increase the statistical strength of individual studies. Meta-analyses are all well and good, but they have their problems too.

I've never fully understood the power-analysis objection. ¿Is it possible you could flesh it out in a layman terms?, ¿Why is it so important?
 
Top