Investigating translation for Indic languages with BLOOMZ-3b through prompting and LoRA fine-tuning

Abstract

Abstract In the domain of natural language processing, the rise of Large Language Models and Generative AI represents a noteworthy transition, enabling machines to understand and generate text resembling that produced by humans. This research conducts a thorough examination of this transformative technology, with a focus on its influence on machine translation. The study explores the translation landscape between English and Indic languages, which include Hindi, Kannada, Malayalam, Tamil, and Telugu. To address this, the Large Language Model, BLOOMZ-3b, is utilized, which has been primarily developed for a text generation task. Multiple prompting engineering techniques for machine translation are prominently explored. The study further traverse fine-tuning the BLOOMZ-3b model using a Parameter Efficient Fine-Tuning technique called Low Rank Adaptation, aiming to reduce computational complexity. Hence, by combining innovative prompting approaches using BLOOMZ-3b model and fine-tuning the model, it contributes to continuous development of machine translation technologies beyond traditional borders of what can be done with respect to language processing. In this regard, not only does this research shed light on the intricacy of translation problems but it also sets a precedence for optimizing or adapting big language models to various languages which end up advancing Artificial Intelligence and Natural Language Processing at large.

Keywords

• Large language models
• Machine translation
• Indic languages
• Prompt engineering
• Parameter efficient fine-tuning
• Low rank adaptation