Probabilistic Thinking Is the Name of the Game: Integrating Test and Confidence Intervals to Plan Sample Sizes

Abstract

Having high statistical power and good estimated precision are essential to statistical practice; however, this integrative consideration on sample size planning remains limited in the literature, especially for two-group mean comparisons with unequal/unknown variances and unequal sampling costs. Furthermore, due to the neglect or misuse of employing confidence intervals, the present study aims to illuminate the probabilistic thinking by finding optimal allocations of sample sizes such that researchers can claim that the null hypothesis is rejected, the desired confidence-interval width of mean difference is achieved, and/or the true difference is encompassed in the interval. Cost effectiveness was also considered to find the optimal sample size. The simulation showed that the proposed approach can maintain the desired probability level for the conditional/unconditional probabilities of events and has good coverage rates in terms of confidence intervals. This study provides an important opportunity to advance the understanding of sample size planning and confidence intervals as well. Three R Shiny apps are provided for easy application in the Supplementary Materials.

Keywords

• desired width
• sampling cost
• unconditional probability
• unequal/unknown variances
• welch’s t test