Some statistical memes which sound correct but aren't quite: Application to the analysis of observational databases used in learning health systems

Abstract

Abstract We consider four memes, correct within the context of randomized trials but requiring modification for the analysis of the observational databases typically associated with learning health systems: (a) the right answer always requires randomization; (b) a bigger database is always a better database; (c) statistical adjustment always works if based on a large enough database; and (d) always make a formal adjustment when testing multiple hypotheses. The rationale for these memes within the context of randomized trials is discussed, and the memes are restated in a fashion that is consistent with learning health systems.

Keywords

• learning health systems
• observational studies
• randomized trials
• statistics