Selection Bias in Real-World Data Studies Used to Support Health Technology Assessments: A Case Study in Metastatic Cancer

Abstract

Real-world evidence has been increasingly used to support evaluations of emerging therapies. These investigations are often conducted in settings that may not be representative of the underlying population. The purpose of this investigation was to empirically quantify the magnitude of this selection bias. Individuals diagnosed with solid metastatic cancer in Alberta, Canada, between 2010–2019 were identified using the provincial cancer registry for 13 common metastatic sites. Two outcomes used to support oncology reimbursement decisions were examined: the proportion of individuals who initiated systemic therapy and median overall survival (OS). These outcomes were assessed in the entire population and in a subset of individuals who were referred to a medical oncologist. Among the 23,152 individuals in the entire population, 40.8% (95% CI: 40.2–41.4) initiated systemic therapy, and the median OS from diagnosis was 5.4 months (95% CI: 5.3–5.6). Among those who were referred to a medical oncologist (n = 13,372; 57.8%), 67.4% (95% CI: 66.6–68.2) initiated systemic therapy, and the median OS from diagnosis was 11.2 months (95% CI: 10.9–11.5). The magnitude of bias varied by cancer site where lower referral rates were associated with greater bias. Non-referral is an important source of selection bias in real-world investigations. Studies that rely on limited-catchment real-world data should be interpreted with caution, particularly in metastatic cancer settings.

Keywords

• real-world evidence
• selection bias
• oncology
• survival
• treatment patterns