BMC Medical Research Methodology (Aug 2024)

The gap between statistical and clinical significance: time to pay attention to clinical relevance in patient-reported outcome measures of insomnia

  • Zongshi Qin,
  • Yidan Zhu,
  • Dong-Dong Shi,
  • Rumeng Chen,
  • Sen Li,
  • Jiani Wu

DOI
https://doi.org/10.1186/s12874-024-02297-0
Journal volume & issue
Vol. 24, no. 1
pp. 1 – 11

Abstract

Read online

Abstract Background Appropriately defining and using the minimal important change (MIC) and the minimal clinically important difference (MCID) are crucial for determining whether the results are clinically significant. The aim of this study is to survey the status of randomized controlled trials (RCTs) for insomnia interventions to assess the inclusion and interpretation of MIC/MCID values. Methods We conducted a cross-sectional study to survey the status of RCTs for insomnia interventions to assess the inclusion and appropriate interpretation of MIC/MCID values. A literature search was conducted by searching the main sleep medicine journals indexed in PubMed, the Excerpta Medica Database (EMBASE), and the Cochrane Central Register of Controlled Trials (CENTRAL) to identify a broad range of search terms. We included RCTs with no restriction on the intervention. The included studies used the Insomnia Severity Index (ISI) or the Pittsburgh Sleep Quality Index (PSQI) questionnaire as the outcome measures. Results 81 eligible studies were identified, and more than one-third of the included studies used MIC/MCID (n = 31, 38.3%). Among them, 21 studies with ISI as the outcome used MIC defined as a relative decrease ranging from 3 to 8 points. The most frequently used MIC value was a 6-point decrease (n = 7), followed by 8-point (n = 6) and 7-point decrease (n = 4), a 4 to 5-points decrease (n = 3), and a 30% reduction from baseline; 6 studies used MCID values, ranging from 2.8 to 4 points. The most frequently used MCID value was a 4-point decrease in the ISI (n = 4). 4 studies with PSQI as the outcome used a 3-point change as the MIC (n = 2) and a 2.5 to 2.7-point difference as MCID (n = 2). 4 non-inferiority design studies considered interval estimation when drawing clinically significant conclusions in their MCID usage. Conclusions The lack of consistent MIC/MCID interpretation and usage in outcome measures for insomnia highlights the urgent need for further efforts to address this issue and improve reporting practices.

Keywords