2023 J. Leonard Goldner Award Winner: Data Censoring Likely Biased the Largest Randomized Trial of Achilles Tendon Rupture Treatment Against Surgery: A Monte Carlo Analysis

Abstract

Category: Sports; Trauma Introduction/Purpose: Randomized controlled trials of surgery versus nonoperative management of Achilles ruptures have been numerous but relatively small. A recent large RCT, published in the New England Journal of Medicine by Myhrvold and coauthors, addressed this with 554 patients enrolled in 3 arms: nonoperative, open surgery, and minimally invasive surgery. The authors found no difference between groups in the primary outcome, the validated Achilles Tendon Rupture Score (ATRS), but a significantly higher rate of rerupture in the nonoperative group (6.2% vs. 0.6% vs 0.6%). The online trial protocol indicates that patients who suffered a rerupture were excluded from further data collection and analysis, but those who had surgical infections and nerve injuries were not. This selective data censoring may have biased the trial outcome against surgery. Methods: To determine if the potential effect from excluding reruptures from final analysis was numerically sufficient to alter the trial results, data models were generated by Monte Carlo analysis using the pre- and post-intervention ATRS score means and standard deviations as ground truths. A model was created of all three groups at their original size. The previously excluded rerupture patients were then added back in at an assumed ATRS score. The original statistical methods of the RCT were used to determine any between-group differences. The analysis was repeated 100,000 times for each possible ATRS score for the rerupture patients. Results: The MCID for the ATRS score is 10 points. Repeated trials analysis found that when rerupture patients were added back into the groups at an assumed ATRS score equal to the reported final mean for nonoperatively treated patients (76), the percentage of model trials that had outcomes favoring a surgical group over nonoperative treatment was 7.9%. This baseline model inherently assumes there is no dysfunction associated with a rerupture. When rerupture was assumed to have associated dysfunction, the percentage of model trials that had outcomes favoring surgery to statistical significance increased as the assumed degree of dysfunction was increased: Rerupture ATRS scores at 1 MCID below nonoperative mean (66): 13.7% Rerupture ATRS scores at 2 MCID’s below nonoperative mean (56): 22.0% Conclusion: Rerupture is almost exclusively a complication of nonoperative treatment. By excluding rerupture patients from further data collection and analysis, the protocol eliminated the likely worst outcomes from the nonoperative treatment while retaining the likely worst outcomes from surgery. While the RCT reported no difference between groups at p<.05, Monte Carlo modeling indicates that at reasonable assumed values of ATRS scores for rerupture patients, the effect of excluding rerupture patients from analysis was potentially large enough to bias the primary trial outcome against surgery. Future trials that follow all patients through their entire treatment course regardless of complications will be required.