Selective publication of antidepressant trials and its influence on apparent efficacy: Updated comparisons and meta-analyses of newer versus older trials

Abstract

Background Valid assessment of drug efficacy and safety requires an evidence base free of reporting bias. Using trial reports in Food and Drug Administration (FDA) drug approval packages as a gold standard, we previously found that the published literature inflated the apparent efficacy of antidepressant drugs. The objective of the current study was to determine whether this has improved with recently approved drugs. Methods and findings Using medical and statistical reviews in FDA drug approval packages, we identified 30 Phase II/III double-blind placebo-controlled acute monotherapy trials, involving 13,747 patients, of desvenlafaxine, vilazodone, levomilnacipran, and vortioxetine; we then identified corresponding published reports. We compared the data from this newer cohort of antidepressants (approved February 2008 to September 2013) with the previously published dataset on 74 trials of 12 older antidepressants (approved December 1987 to August 2002). Using logistic regression, we examined the effects of trial outcome and trial cohort (newer versus older) on transparent reporting (whether published and FDA conclusions agreed). Among newer antidepressants, transparent publication occurred more with positive (15/15 = 100%) than negative (7/15 = 47%) trials (OR 35.1, CI95% 1.8 to 693). Controlling for trial outcome, transparent publication occurred more with newer than older trials (OR 6.6, CI95% 1.6 to 26.4). Within negative trials, transparent reporting increased from 11% to 47%. We also conducted and contrasted FDA- and journal-based meta-analyses. For newer antidepressants, FDA-based effect size (ESFDA) was 0.24 (CI95% 0.18 to 0.30), while journal-based effect size (ESJournals) was 0.29 (CI95% 0.23 to 0.36). Thus, effect size inflation, presumably due to reporting bias, was 0.05, less than for older antidepressants (0.10). Limitations of this study include a small number of trials and drugs—belonging to a single class—and a focus on efficacy (versus safety). Conclusions Reporting bias persists but appears to have diminished for newer, compared to older, antidepressants. Continued efforts are needed to further improve transparency in the scientific literature. Author summary Why was this study done? ➢ Clinicians and researchers depend on the peer-reviewed literature for accurate assessments of drug efficacy and safety, but this depends on whether the outcomes of all trials—negative, as well as positive—are reported transparently. ➢ In an earlier study, using Food and Drug Administration (FDA) review documents as a gold standard, we found that many negative trials had been misreported in the published literature as having positive outcomes or had simply not been published. ➢ Since then, reporting bias has been the subject of additional studies and policy changes, raising the question, Is the antidepressant literature now being reported more transparently? What did the researchers do and find? ➢ Using FDA reviews on 4 newer antidepressants, we identified 30 trials, half with positive, and half with negative, outcomes. ➢ Among the 15 negative trials, 6 were unpublished and 2 others were misreported as positive. Seven other negative trials (47%) were reported transparently (as negative), an improvement over the low (11%) rate found earlier with the older antidepressants. ➢ Statistical comparison of the newer and older drug datasets indicated that transparent reporting had improved overall, mainly among negative trials. Yet compared to positive trials, the rate of transparent reporting for negative trials remains low. ➢ Using meta-analysis to compare drug efficacy based on FDA versus published data, we found less inflation of drug efficacy among newer, compared to older, antidepressants. What do these findings mean? ➢ Reporting bias persists but appears to have diminished for newer, compared to older, antidepressants. ➢ We do not know whether these results extend to drugs beyond the antidepressants studied here, nor do we know whether they extend to drug safety, as opposed to efficacy. ➢ Reporting bias remains a significant impediment to researchers and medical decision-makers, so further efforts are needed to improve transparent reporting in the scientific literature.