Multimodal Machine Translation Approaches for Indian Languages: A Comprehensive Survey

Abstract

Multimodal machine translation (MMT) is a challenging task in the linguistically diverse Indian landscape. Machine translation refers to the task of automatically converting content from one language to another without human involvement. Within the realm of natural language processing, a significant challenge arises from the inherent ambiguity present in human language. Translation ambiguity is a cross-lingual phenomenon that can manifest itself for various reasons, including lexical ambiguity, the occasional need to impute missing words, the presence of gen-der ambiguity, and word-sense ambiguities. These factors can lead to a decrease in translation accuracy. The integration of multiple modalities, such as images, videos, and audio, in addition to text, plays a pivotal role in improving the robustness and precision of translation systems. Over the past five years, extensive research has been dedicated to incorporating secondary modalities alongside text to improve language translation and comprehension. In this comprehensive study, our objective was to identify and explore promising MMT approaches, available corpora, eval-uation metrics, research challenges, and the future direction of research specifically for Indian languages. We evaluated 81 papers, including MMT models, MMT dataset in Indian languages, survey on MMT approach, and the effects of multiple modalities in machine translation. The performance of the different proposed approaches has also been briefly analyzed on the basis of the claimed results and comparative evaluations. Finally, the challenges associated with the MMT task for India and some possible directions for future research in this domain are highlighted.

Keywords

• Multimodal Machine Translation
• Multimodal data
• M