BERT‐TriF: An inductive short text classification model for power equipment defect records

Abstract

Abstract The descriptions of power equipment defect records are often characterized by colloquial short texts. Standardized classification of a large number of colloquial defect descriptions has laid a solid foundation for building a power equipment knowledge graph and improving the level of intelligence in the field of power inspection. Using deep learning and natural language processing technology, this paper proposes a text classification model for power equipment defect records named Bidirectional Encoder Representations from Transformers with Family Feature Fusion (BERT‐TriF). The model firstly leverages BERT to semantically represent the input text. To extract family history as well as text implicit information, we creatively propose a family feature fusion algorithm for training. An improved multi‐head attention mechanism is developed subsequently to enhance text semantic category features and strengthen the learning ability of the model. By comparing BERT‐TriF and baseline models such as TextCNN, TextRNN, and fastText on the specified and generic text dataset, the experimental results demonstrate that it has better performance, robustness, and universality for short text classification.

Keywords

• family feature fusion
• multi‐head attention mechanism
• power equipment defect record
• short text classification