Abstract Background Quality measurement has become a priority for national healthcare reform, and valid measures are necessary to discriminate hospital performance and support value‐based healthcare delivery. The Commission on Cancer (CoC) is the largest cancer‐specific accreditor of hospital quality in the United States and has implemented Quality of Care Measures to evaluate cancer care delivery. However, none has been formally tested as a valid metric for assessing hospital performance based on actual patient outcomes. Methods Eligibility and compliance with the Quality of Care Measures are reported within the National Cancer Database, which also captures data for robust patient‐level risk adjustment. Hospital‐level compliance was calculated for the core measures, and the association with patient survival was tested using Cox regression. Results Seven hundred sixty‐eight thousand nine hundred sixty‐nine unique cancer cases were included from 1323 facilities. Increasing hospital‐level compliance was associated with improved survival for only two measures, including a 35% reduced risk of mortality for the gastric cancer measure G15RLN (HR 0.65, 95% CI 0.58–0.72) and a 19% reduced risk of mortality for the colon cancer measure 12RLN (HR 0.81, 95% CI 0.77–0.85). For the lung cancer measure LNoSurg, increasing compliance was paradoxically associated with an increased risk of mortality (HR 1.14, 95% CI 1.08–1.20). For the remaining measures, hospital‐level compliance demonstrated no consistent association with patient survival. Conclusion Hospital‐level compliance with the CoC’s Quality of Care Measures is not uniformly aligned with patient survival. In their current form, these measures do not reliably discriminate hospital performance and are limited as a tool for value‐based healthcare delivery.