Based on Gated Dynamic Encoding Optimization, the LGE-Transformer Method for Low-Resource Neural Machine Translation

Abstract

In current Neural Machine Translation (NMT) research, translating low-resource language pairs remains a significant challenge. This work proposes an LGE-Transformer method for Chinese-Malay neural machine translation based on Gated Dynamic Encoding Optimization. By introducing a linguistically enhanced LERT pre-training model, the Transformer encoder is reconstructed and integrated with a gated dynamic encoding module, effectively integrating features from various encoder layers and enhancing the model’s representation capability in low-resource language pairs. On the encoder side, the proposed method achieves an adaptive fusion of multi-layer encoder outputs through the gated dynamic encoding module, enabling the model to fully utilize feature information from all layers, thereby improving translation accuracy and fluency. On the decoder side, we introduce a hybrid cross-attention module, further enhancing the model’s attention to contextual information, and thereby improving the semantic accuracy of the translation results. Experimental results on the Chinese-Malay low-resource translation task demonstrate that the proposed LGE-Transformer method significantly outperforms the baseline and other experimental models in terms of BLEU scores, validating the effectiveness and superiority of the gated dynamic encoding optimization-based neural machine translation method in low-resource language pair translation tasks.

Keywords

• Pretraining language model
• GRU gating
• Chinese-Malay machine translation
• low resource
• deep learning