AI-based translation quality for low-resource languages : the case of Maltese

Abstract

Maltese is the national language of the Republic of Malta and shares its official language status with English. Despite being used by a small community mostly in oral communication and in a few formal domains, it became an official language of the European Union in 2004. In the last two decades, because of EU membership, translation requirements in Maltese have greatly increased. However, the relative scarcity of Maltese texts in many domains, especially in technical fields, has created a critical data deficiency for training large language models (LLMs). Hence, the quality of AI-driven translation for Maltese is generally perceived to be inadequate. However, to date, no studies have been made to assess translation quality related to the use of AI-based technologies for the Maltese language. To address this research gap, the present small-scale study evaluates the performance of two prominent AI-based translation tools, Google Translate and ChatGPT, on a 6000-word corpus of 20 texts translated from Italian into both Maltese and English. The raw output was systematically evaluated using an adapted DQF-MQM error typology template. The results show that in the case of Maltese, Google Translate made almost three times more errors with respect to English, while ChatGPT generated over seven times the errors for Maltese. The analysis concludes that despite the high status of Maltese in the EU’s multilingual setting, the limitations of Maltese as a low-resource language still persist, and a highly cautious approach must be taken by Maltese translators and post-editors when using AI-based tools for translation.