From: utzoo!henry Newsgroups: net.misc Title: Re: Re: ESP & Elementary Probability Article-I.D.: utzoo.2848 Posted: Tue Mar 1 16:29:01 1983 Received: Tue Mar 1 16:29:01 1983 "Most researchers do not claim "truth" based on a single experiment unless the probability is extremely small (p < 10e-9 say). The usual process is to look at other independent tests, calculate the overall statistic, and then make a judgement. Most laymen are statistically naive..." Researchers who follow this procedure are also statistically naive. It does not matter how many existing results your theory can explain; if it cannot *predict* new results, then what you are doing is not science. It's easy to fiddle with a theory until it explains data you already know about; the acid test is new data, not considered in the formulation of the theory. Obviously, this causes trouble if you aren't sure what you're after. There are ways around this, but only the more conscientious parapsy- chology researchers use them. Said conscientious researchers are also seldom heard claiming positive results. In case people are curious, at least one of said methods is very simple. You divide your data into two halves along some arbitrary line, agreed on before the experiment. (E.g., first n/2 tests and last n/2 tests.) You put the second half away and don't look at it. You then examine the first half in any way you please, looking for correlations. Once you have determined what phenomenon appears to be manifesting itself, you settle on a specific data-analysis procedure and specific criteria for success and failure. Then, you dig out the second half of the data and apply that exact procedure and those exact criteria. You are not allowed to fiddle with the analysis method once you have started to examine the second half. If you get positive results on the second half, using the method and criteria derived from the first half, THEN you have a positive result for the experiment. If not, you have a negative result and no further fudging is allowed. Show me positive results obtained with this method, or some other way of making sure you don't outwit yourself, and I will be impressed. Statistical analysis of parapsychology data without such safeguards is worthless. Henry Spencer U of Toronto