Estimation statistics should replace significance testing
In place of significance testing, a preferred statistical methodology is accessible with modest re-training. However, an obstacle to the adoption of this alternative is a basic branding problem: it does not have a widely-used name. We suggest the most appropriate name for this superior approach is ‘estimation statistics,’ a term describing the methods that focus on the estimation of effect sizes (point estimates) and their confidence intervals (precision estimates). Estimation statistics offers several key benefits.
This letter was previously published In Nature Methods at http://dx.doi.org/10.1038/nmeth.3729
Altman, D., D. Machin, T. Bryant, and S. Gardner. 2000. Statistics with Confidence: Confidence Interval and Statistical Guidelines. Bristol: BMJ Books.
Cohen, Jacob. 1994. "The Earth Is Round (p < .05)." The American Psychologist 49 (12): 997–1003.
Cumming, Geoff. 2012. Understanding the New Statistics Effect Sizes, Confidence Intervals, and Meta-Analysis. New York: Routledge.
Halsey, Lewis G., Douglas Curran-Everett, Sarah L. Vowler, and Gordon B. Drummond. 2015. "The Fickle P Value Generates Irreproducible Results." Nature Methods 12 (3): 179–85.
Horton, Richard. 2015. "Offline: What Is Medicine's 5 Sigma?" The Lancet 385 (9976). Elsevier: 1380.
Morrison, Denton E., and Ramon E. Henkel. 2006. The Significance Test Controversy: A Reader. AldineTransaction.
Nuzzo, Regina. 2015. "Scientists Perturbed by Loss of Stat Tools to Sift Research Fudge from Fact." Scientific American, April 16.
Sena, Emily S., H. Bart van der Worp, Philip M. W. Bath, David W. Howells, and Malcolm R. Macleod. 2010. "Publication Bias in Reports of Animal Stroke Studies Leads to Major Overstatement of Efficacy." PLoS Biology 8 (3): e1000344.
Tukey, John W. 1969. "Analyzing Data: Sanctification or Detective Work?" The American Psychologist 24 (2). American Psychological Association: 83.
Vesterinen, H. M., E. S. Sena, K. J. Egan, T. C. Hirst, L. Churolov, G. L. Currie, A. Antonic, D. W. Howells, and M. R. Macleod. 2014. "Meta-Analysis of Data from Animal Studies: A Practical Guide." Journal of Neuroscience Methods 221 (January): 92–102.
Yildizoglu, Tugce, Jan-Marek Weislogel, Farhan Mohammad, Edwin S-Y Chan, Pryseley N. Assam, and Adam Claridge-Chang. 2015. "Estimating Information Processing in a Memory System: The Utility of Meta-Analytic Methods for Genetics." PLoS Genetics 11 (12): e1005718.