Unsupervised Grammatical Correction With Optimized Layer Normalization and Dynamic Embedding Enhancement

Abstract

Grammatical error correction aims to detect and correct grammatical errors with all types of mistaken, disordered, missing, and redundant characters. However, most existing methods focus more on detecting errors than correcting them. This paper proposes a domain-adaptive model with Interoperable Layer Normalization (ILN) and dynamic word embedding enhancement to optimize the error correction capability. To further improve the Chinese correction capability, we introduce multiple rounds of error correction to refine the sequence tagging model’s ability to fix mistakes. In addition, we propose a data augmentation method based on the complex tag to represent textual error correction traces more completely. We also explore a migration training method based on multiple training datasets. Further, we offer a unique unsupervised domain adaptation technique based on ILN, an innovative channel fusion approach that can significantly improve models’ domain adaptability. Finally, experimental results show that our proposed method substantially outperforms all robust baseline methods and achieves the best results in position-level and correction-level errors on the CGED-2020 dataset.

Keywords

• Grammatical correction
• layer normalization
• dynamic embedding enhancement