Network Biology (Sep 2022)
Confidence intervals: Concepts, fallacies, criticisms, solutions and beyond
Abstract
For a long time, confidence interval theory is the basis of statistics, and confidence interval has been regarded as an important content of statistical analysis. Almost all statistical textbooks and statistical analysis software contain the contents of confidence intervals, which are used to estimate statistical parameters or parameters of mathematical models, and are an important part of many methods such as interval estimation, analysis of variance, and regression analysis, etc. They are recommended or required by the method guidelines of many reputable journals. So far, confidence interval theory and methods have been widely used in various scientific or engineering fields including life sciences, medicine, environmental science, chemistry, physics, and psychology. However, due to the fallacies or deficiencies of the confidence interval theory and methodology, it has caused a wide range of misuses, and has been criticized more and more in recent years. Some statisticians even suggest abandoning the confidence interval theory. To avoid the problems of classical confidence interval theory, one can use Bayesian credible intervals, use uncertainty methods, calculate confidence intervals by avoiding statistic significance tests, or use the Bootstrap credible interval method proposed by me, etc. In practice, for controlled experiments, multiple replicates or treatments should be designed; for observational experiments, multiple representative samples should be drawn, and even a single sample can be used if sufficient sample size is ensured. It is necessary to implement the whole process control for every procedures from sampling to statistical analysis. Cross-comparison and validation of confidence interval analysis results with other multi-source results should be conducted to obtain the most reliable conclusions. Finally, in addition to writing, publishing and adopting new statistical works and teaching materials as soon as possible, it is imperative to revise and distribute various statistical software in new editions based on new statistics for use.
