Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 9/18/84; site LaBrea.ARPA Path: utzoo!watmath!clyde!bonnie!akgua!whuxlm!harpo!decvax!decwrl!Glacier!LaBrea!mann From: mann@LaBrea.ARPA Newsgroups: net.physics Subject: Re: Parapsychology Experiment Now Message-ID: <123@LaBrea.ARPA> Date: Sat, 6-Jul-85 00:37:14 EDT Article-I.D.: LaBrea.123 Posted: Sat Jul 6 00:37:14 1985 Date-Received: Fri, 12-Jul-85 02:48:45 EDT References: <2896@decwrl.UUCP> Organization: Stanford University Lines: 23 > The highest score over the 95% confidence level will be awarded a > tasteful brass plaque. As I'm sure you recognize, if you get 100 responses, the expected number of scores that are significantly high at the 95% confidence level is 5, even if no one has any ESP at all. So I hope you have a good source for a cheap (but tasteful) brass plaque -- you'll be awarding it. More seriously, this points up a real pitfall that many ESP researchers have fallen into. They know enough about statistics and scientific practice to get excited about a result that is "significant at the 95% level", and even more excited about one that's significant at the 99% level. So they do hundreds of experiments, and test each one's significance individually. Lo and behold, about 1 in 100 experiments is "significant" at the 99% level, so they report it. Of course, this is just what would be expected by chance. "Significant at the 99% level" means there is only 1 chance in 100 of the given outcome having occured entirely by chance. If most or all your experiements are significant at this level, or (far better) if all your experiments taken together are significant at this level, you are really on to something. If only 1 in 100 is "significant", you have merely observed the laws of probability in action. --Tim