posted by Kevin on .
Carol has data on the causes of violent outbursts among kindergarten children. She used a "shot-gun" approach to hypothesis testing by asking the computer to produce p-values for 300 quickly chosen hypothesis tests. She shared her glee with fellow researchers, noting that she had found 15 statistically significant results. She went so far as to brag that she had made some discoveries that defied all previously held notions about the wild behavior of this population of terrifying tots. She used the .05 level of significance. Explain why she actually found nothing.
.05 * 300 = 15
This means you would expect 15 tests to be significant solely by chance.