A while ago I found a very interesting paper from Leah R. Jager and Jeffrey T. Leek via a post in the Simply Statistics blog arguing that most published medical research is true with a rate of false positives among reported results of 14% ± 1%. Their paper came as a response to an essay from John P. A. Ioannidis and several others authors claiming that most published research findings are false.
After dealing with some criticisms Mr. Leek made a good point in his post:
“I also hope that by introducing a new estimator of the science-wise fdr we inspire more methodological development and that philosophical criticisms won’t prevent people from looking at the data in new ways.”
And thus, following this advice, I didn’t let criticisms prevent me from looking at the data in a new way. So for this problem I have devised a probability distribution for p-values to then fit the data via MLE and infer from there the rate of false positives.
So this is my take; 15.33% rate of false positive with a worse case scenario of 41.75% depending on how mischievous researchers are but, in any case, and contrary to what others authors claim, most medical research seems to be true.
Alea Deum 