A retrospective assessment of marine ecological research using optimal α
Loading...
Files
Date
2020
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
University of New Brunswick
Abstract
Null hypothesis significance testing (NHST) is and has been for many decades broadly applied across disciplines, NHST’s shortcomings have been identified and discussed since its introduction. Optimal alpha is a better method for setting statistical thresholds in NHST because it minimizes the overall probability of making errors. However, we don’t know how often using optimal α would result in a different conclusion than NHST. I calculate optimal α for 433 tests from 2009-2018 published marine biology papers and compare conclusions with NHST. I find totally 24 % of conflicting results (small ES: 22 %; medium ES: 22 %; large ES: 29 %). For disagreement, optimal alpha has 97 % significant results at small ES, 54% at medium and 24% at large. Low observed p value tends to have disagreement. These results mean that failing to use optimal alpha may be inflating the probability of making wrong conclusions in marine biology.