Attention-based interactive multi-level feature fusion for named entity recognition

Abstract

Abstract Named Entity Recognition (NER) is an essential component of numerous Natural Language Processing (NLP) systems, with the aim of identifying and classifying entities that have specific meanings in raw text, such as person (PER), location (LOC), and organization (ORG). Recently, Deep Neural Networks (DNNs) have been extensively applied to NER tasks owing to the rapid development of deep learning technology. However, despite their advancements, these models fail to take full advantage of the multi-level features (e.g., lexical phrases, keywords, capitalization, suffixes, etc.) of entities and the dependencies between different features. To address this issue, we propose a novel attention-based interactive multi-level feature fusion (AIMFF) framework, which aims to improve NER by obtaining multi-level features from different perspectives and interactively capturing the dependencies between different features. Our model is composed of four parts: the input, feature extraction, feature fusion, and sequence-labeling layers. First, we generate the original word- and character-level embeddings in the input layer. Then, we incorporate four parallel components to capture global word-level, local word-level, global character-level, and local character-level features in the feature extraction layer to enrich word embeddings with comprehensive multi-level semantic features. Next, we adopt cross-attention in the feature fusion layer to fuse features by exploiting the interaction between word- and character-level features. Finally, the fused features are fed into the sequence labeling layer to predict the word labels. We conducted generous comparative experiments on three datasets, and the experimental results showed that our model achieved better performance than several state-of-the-art models.

Keywords

• Named entity recognition
• Multi-level features
• Cross-attention
• Feature fusion