Survey of Neural Machine Translation Based on Knowledge Distillation

Abstract

Machine translation (MT) is the process of using a computer to convert one language into another language with the same semantics. With the introduction of neural network, neural machine translation (NMT), as a powerful machine translation technology, has achieved remarkable success in the field of automatic translation and artificial intelligence. Due to the problem of redundant parameters and structure in traditional neural translation models, knowledge distillation (KD) technology is proposed to compress the model and accelerate the inference of neural machine translation, which has attracted wide attention in the field of machine learning and natural language processing. This paper systematically investigates and compares various translation models with introduction of  know-ledge distillation from the perspectives of evaluation indicators and technical innovations. Firstly, this paper briefly reviews the development process, mainstream frameworks and evaluation indicators of machine translation. Secondly, the knowledge distillation technology is introduced in detail. Thirdly, the development direction of neural machine translation based on knowledge distillation is detailed from four perspectives: multi-language model, multi-modal translation, low-resource language, autoregressive and non-autoregressive, and the research status of other fields is briefly introduced. Finally, the problems of existing large language models, zero-resource languages and multi-modal machine translation are analyzed, and the development trend of neural machine translation is prospected.

Keywords

• machine translation; neural machine translation; knowledge distillation; model compression