Saturday, July 19, 2014

Concensus and Sigma

Consensus in science is very reassuring to the outsider.

Sigma is the Greek letter used as a symbol for standard deviation—a measure of how far a finding departs from the expected one. Say a series of experiments on a randomly chosen coin involves flipping it 1,000 times and then flipping it 1,000 times again and again. The average number of heads should be 500, but some experiments will yield more and some fewer. A five-sigma finding would be 590 heads. Hence the safety and quality of "Six Sigma."
In IQ testing the expected result is an IQ of 100. One-Sigma deviation from that would be about 115 (or 85), so two- sigma deviation would be 130, 3-Sigma 145. This means that when testing the IQ of a population, the chance of one individual having an IQ of 70 would be 2 Sigma, about 5%
File:IQ curve.svg

So a 5 Sigma IQ is not impossible, only very unlikely. But, with a population of 3.5 million, a random IQ test could make you think that the outrageous outlier you get is the norm.


In theoretical physics, it is more difficult because you are not measuring the chances of deviation off the norm, you are measuring the possibility of existence.

Recent testing for the Higgs Particle  was done with a 5-Sigma probability. The five-sigma concept is somewhat counterintuitive. It has to do with a one-in-3.5-million probability--but not the probability that the Higgs boson doesn't exist. It is, rather, the inverse: If the particle doesn't exist, one in 3.5 million is the chance an experiment just like the one announced would nevertheless come up with a result appearing to confirm it does exist. In other words, one in 3.5 million is the likelihood of finding a false positive. So the test, theoretically, could confirm the existence of something that does not exist.

A two-sigma result can have as much as a 5% chance of occurring as a false positive. Three sigma, needed to cite evidence—but not discovery—of a new particle in physics, correspond to a one-in-741 chance.

So any theory where there is "consensus" should be qualified with "probably."

The real challenge is designing a study in which the structure of the information gathering is, indeed, rigid enough to merit the probability designated to it.

No comments: