Statcheck Checks PubPeer Stats and Conclusions

“When starting this project, I wouldn’t say [this was a big problem],” Nuijten tells me. “We’re detecting when people are making rounding errors, who cares?”

But she and some colleagues in the Netherlands were curious enough to check. They built a computer program that could quickly scan published psychological papers and check the math on the statistics. They called their program “Statcheck” and ran it on 30,717 papers.

Rounding errors, and other small potential mistakes in calculating the statistics, were rampant. “We found that half of all published psychology papers … contained at least one p-value that was inconsistent with its test,” Nuijten and her co-authors reported in 2015 in the journal Behavior Research Methods.

Most striking was that the errors weren’t entirely random. Most of the errors tipped the results in favor of statistical significance. And around 13 percent of the papers contained an error that could potentially change the paper’s conclusions.

–Shamelessly stolen from Vox.com