An ELECTRA-Based Model for Power Safety Named Entity Recognition

Abstract

Power safety named entity recognition (NER) is essential for determining the cause of faults, assessing potential risks, and planning maintenance schedules, contributing to the comprehension and analysis of power safety documentation content and structure. Such analysis is crucial for the development of a knowledge graph within the power safety domain and the augmentation of the associated dataset. This paper introduces a power safety NER model using efficiently learning an encoder that classifies token replacements accurately (ELECTRA) model. This model employs root mean square layer normalization (RMSNorm) and the switched gated linear unit (SwiGLU) activation function, which substitutes the conventional layer normalization (LayerNorm) and the Gaussian error linear units (GeLU). This model also integrates bidirectional long short-term memory (BiLSTM) with conditional random fields (CRF) to bolster performance in NER tasks. Experimental results show that the improved ELECTRA model achieved an F1 value of 93% on the constructed power safety NER dataset. It outperforms the BERT-BiLSTM-CRF model, achieving a 3.3% performance improvement.

Keywords

• named entity recognition
• power safety data
• ELECTRA
• SWiGLU