An Attentive Neural Sequence Labeling Model for Adverse Drug Reactions Mentions Extraction

Abstract

Adverse drug reactions (ADRs) are a main cause of morbidity and mortality in patients. Extracting mentions of ADRs from the health-related text has important applications in biomedical research. Existing work mainly utilizes feature-based pipeline methods or neural network models that use only word embeddings as input features. These methods either require many efforts to design task-specific features or suffer misclassification on those words, which have not been seen before. Therefore, we propose an end-to-end neural sequence labeling model that labels words in an input sequence with ADRs membership tags. In addition to word-level embeddings, we also adopt character-level embeddings and combine them via an embedding-level attention mechanism. Through such an attention mechanism, our model can dynamically determine how much information to utilize from a word- or character-level component. In addition, we use the intermediate output of the model as an auxiliary classifier and combine it with the final output of the model to improve the overall performance. We evaluate different architectures on two ADRs labeling datasets. One is an ADRs-related Twitter corpus that includes many informal vocabularies and irregular grammar, and the other is a biomedical text extracted from PubMed abstracts with many professional terms and technical descriptions. Our model achieves approximate match F1 scores of 0.844 and 0.906 for ADRs identification on the Twitter and PubMed datasets, respectively. It presents the state-of-the-art performance on both the datasets. Our system is completely end-to-end, requires no task-specific feature engineering or hand-crafted features, and thus can be generalized to a wide range of sequence labeling tasks.

Keywords

• Adverse drug reactions
• biomedical text
• deep learning
• neural network
• sequence labeling
• social media