Evaluation of quantitative signal detection in EudraVigilance for orphan drugs: possible risk of false negatives

Abstract

Different strategies have been studied to allow a better characterization of the safety profile of orphan drugs soon after their approval. At the end of the development phases only few data are available because of the small number of subjects exposed to an orphan medicine for the treatment of rare or ultra-rare conditions. As a consequence, the evaluation of the safety profile is limited at the time of the first approval. In the post-marketing period, all available sources should be combined for a better understanding of the safety of orphan drugs. These sources, include outputs from large databases such as the European Medicines Agency’s EudraVigilance database. Analyses of data from this source are required to be performed by marketing authorization holders (MAHs) as part of their signal management activities. In 2018, the Pharmacovigilance Risk Assessment Committee (PRAC) assessed 114 confirmed signals, 79% of which included data from EudraVigilance. MAHs have access to statistical calculations for drug–event combinations (DECs) from EudraVigilance, provided in the form of measures of disproportionality of ratios of the observed proportion of spontaneous cases for a DEC in relation to the proportion of cases that would be expected if no association existed between the drug and the event. However, such statistical summaries for orphan drugs could be misleading because of the very limited safety data available for orphan drugs (under-reporting together with low numbers of exposed patients). In addition, the applied statistical methodology in most instances is constrained by different confounding factors such as indications of specific medicines and the wide spectrum of medical conditions/diseases of patients from whom reporting of disproportionality ratios are derived (i.e. proportions of DECs for orphan drugs (ODECs) from a small patient population suffering the rare disease and the proportion of DECs in the rest of the population represented in the whole database who have been treated with other medicines for a wide range of indications, and prescribed to treat completely different medical conditions). As expected, these statistical calculations produced not only signals of disproportionate reporting (SDRs) that are false positives, but also not sensitive enough to detect certain SDRs, thus resulting in false negatives. In the context of rare/ultra-rare life-threatening diseases where new molecules have been made available on the market on the basis of their proven efficacy, but with only limited safety data at the time of approval, false negatives could be a special concern since unlikely converted in positives or becoming positives with notable delay. Subgroup analyses (using a limited dataset comprising ADRs within specific individual case safety reports (ICSRs), sorted by indication/disease relevant to the drug of interest could, at least in part, possibly reduce some of the weaknesses resulting from the abovementioned confounding factors. On the other hand it could also cause the loss of some identification of SDRs that would be captured if no database restrictions had been undertaken. Therefore, data subgroup analysis should not be selected as a preferred approach to quantitative signal detection for orphan drugs but rather evaluated as complementary possibly to confirm negatives or to further characterize detected SDRs. Some examples of false negatives originating from quantitative signal detection in EudraVigilance applied to orphan drugs are discussed in this article.