A Distributed Knowledge Distillation Framework for Financial Fraud Detection Based on Transformer

Abstract

Financial fraud cases causing serious damage to the interests of investors are not uncommon. As a result, a wide range of intelligent detection techniques are put forth to support financial institutions’ decision-making. Currently, existing methods have problems such as poor detection accuracy, slow inference speed, and weak generalization ability. Therefore, we suggest a distributed knowledge distillation architecture for financial fraud detection based on Transformer. Firstly, the multi-attention mechanism is used to give weights to the features, followed by feed-forward neural networks to extract high-level features that include relevant information, and finally neural networks are used to categorize financial fraud. Secondly, for the problem of inconsistent financial data indicators and unbalanced data distribution focused on different industries, a distributed knowledge distillation algorithm is proposed. This algorithm combines the detection knowledge of the multi-teacher network and migrates the knowledge to the student network, which detects the financial data of different industries. The final experimental results show that the proposed method outperforms other methods in terms of F1 score (92.87%), accuracy (98.98%), precision (81.48%), recall (95.45%), and AUC score (96.73%) when compared to the traditional detection methods.

Keywords

• Transformer
• knowledge distillation
• financial fraud detection