TQFLL: a novel unified analytics framework for translation quality framework for large language model and human translation of allusions in multilingual corpora

Abstract

In large language models (LLMs), the translation quality has limitations in the translation when translated into different languages. This study compares Chinese allusions in human and machine translated corpora translated by OpenAI GPT-3.5, Volctrans, and human translated texts. The framework innovatively combines two automated evaluation metrics, BLEU and METEOR, with a translation quality assessment method derived from Fuzzy Mathematics and Optimality Theory. The findings of the study indicate that the GPT-3.5 translated version exhibits higher quality than the Volctrans version when evaluated by a machine. Similarly, human evaluations indicate that among the three versions, Volctrans is of the lowest quality, while the human version exceeds the GPT-3.5 version in terms of quality. Thus, the study further reveals that Volctrans version is deemed to have the lowest translation quality from both human and machine evaluation perspectives. Finally, this study not only introduces but also validates a novel framework for assessing machine translation quality.

Keywords

• LLMs
• multilingual corpora
• translation quality assessment
• machine translation
• allusions