Automatic Post-editing of Hierarchical Attention Networks for Improved Context-aware Neural Machine Translation

Abstract

Most of the existing neural machine translation (NMT) methods translate sentences without considering the context. It is shown that exploiting inter and intra-sentential context can improve the NMT models and yield to better overall translation quality. However, providing document-level data is costly, so properly exploiting contextual data from monolingual corpora would help translation quality. In this paper, we proposed a new method for context-aware neural machine translation (CA-NMT) using a combination of hierarchical attention networks (HAN) and automatic post-editing (APE) techniques to fix discourse phenomena when there is lack of context. HAN is used when we have a few document-level data, and APE can be trained on vast monolingual document-level data to improve results further. Experimental results show that combining HAN and APE can complement each other to mitigate contextual translation errors and further improve CA-NMT by achieving reasonable improvement over HAN (i.e., BLEU score of 22.91 on En-De news-commentary dataset).

Keywords

• context-aware neural machine translation
• document-level neural machine translation
• neural machine translation