Translation product evaluation in translator training : a comparison between L1 (English > Dutch) and L2 (Dutch > English) student translations using the PIE method (preselected items evaluation) and the ATA (American translators association) framework for standardized error marking

Abstract

Translation evaluation is a contentious area of study, particularly due to the degree of subjectivity of many evaluation methods, yet it is essential in translator training. Likewise, directionality is a controversial field due to the hegemony of translation into the first language (L1 translation). Nevertheless, translation into the foreign language (L2 translation) is included in most, if not all, translator training programmes in many countries.

The present study combines these two much debated fields of study into an empirical experiment conducted in translator training in Flanders. Its aim is twofold. Firstly, it provides insight into the differences between L1 and L2 translation in terms of scores and error types on the basis of the data for this study and a prior pilot study. It also compares the perceived difficulty of both translation directions with the scores achieved. From a process-oriented perspective, this study compares the time spent on the translations with the final scores. The second aim of the study is to scrutinise the evaluation process and empirically test a combined analytical evaluation method consisting of an enhanced version of PIE (Preselected Items Evaluation), an item-based analytical evaluation method, and the ATA (American Translators Association) Framework for Standardized Error Marking.

The findings reveal that, while the scores in both translation directions are low and comparable, surprisingly, the students made slightly more errors in L1 translation, despite perceiving the L2 translation as more difficult. In both translation directions, the main errors identified are terminology errors. In addition, both L1 and L2 translation are characterised by source language interference. It was not possible to establish a conclusive relation between the time spent on the translations and the final scores. As for the evaluation process, the present study reveals several limitations of PIE and the combined PIE and ATA method, and calls for further research into the application of these evaluation methods, particularly into the intersubjective consensus.

It is hoped that this study will inspire L1 and L2 translation lecturers in translator training to consider the evaluation methods they use carefully and refine the L2 translation curriculum based on solid evaluations and contrastive empirical research.