MTPEAS in the Translation Classroom: A Mixed-Methods Study With an Eye on the Future
DOI:
https://doi.org/10.17507/jltr.1606.23Keywords:
MTPEAS, MT, post-editing, taxonomy, translation qualityAbstract
The growing use of machine translation (MT) for both academic and professional translation purposes requires uniformly accepted frameworks for post-edited outputs to be assessed. The current research investigates the application of the Machine Translation Post-Editing Annotation System (MTPEAS) in pedagogy for translation, suggesting a systematic taxonomy for determining the quality of students' post-edited work. In particular, the study seeks to explore the efficacy of MTPEAS in enhancing the quality of translation, assessing major post-editing performance measures, increasing students' attitudes towards using MTPEAS, and exploring teaching problems related to using it in the classroom. Employing a mixed-methods design, the study examines pre-and post-test data from 150 university students as well as instructor feedback interviews with 10 instructors. Quantitative data such as means and percentages were employed to evaluate skill development and attitude of Saudi students towards MTPEAS, and questionnaire findings clarify pedagogical and operational challenges. Conclusions seek to bridge the gap between industry expectations and translation education by illustrating how MTPEAS may normalize assessment, detect gaps in learning, and improve post-editing skills. MTPEAS developed translation training through engaging students, encouraging error awareness, and allowing formative assessment. The usability issues, complicated terminology, and inadequate training necessitate MTPEAS's improvements.
References
Al-Ahdal, A. A. M. H., Alfallaj, F. S. S., Al-Awaid, S. A. A., & Al-Mashaqba, N. J. A. H. (2017). Translation courses at Qassim University, Saudi Arabia: A study of existing problems and possible solutions. US-China Foreign Language, 15(3), 45-53.
Alowedi, N. A., & Hassan Al Ahdal, A. A. M. (2023). Artificial Intelligence based Arabic-to-English machine versus human translation of poetry: An analytical study of outcomes. Journal of Namibian Studies, 33, 1523-1538.
Bartaškevičius, O. (2024). Light machine translation post-editing effort and quality evaluation of news texts translated from English to Lithuanian [Doctoral dissertation]. Kauno technologijos universitetas.
Bodart, R., Piette, J., & Lefer, M. A. (2024). The machine translation post-editing annotation system (MTPEAS) A standardized and user-friendly taxonomy for student post-editing quality assessment. Translation Spaces, 13(2), 265-292. https://doi.org/10.1075/ts.24002.bod
Cohen, J. (1988). The effect size. Statistical power analysis for the behavioral sciences. Abingdon: Routledge, 77-83.
Esfandiari, M. R., Shokrpour, N., & Rahimi, F. (2019). An evaluation of the EMT: Compatibility with the professional translator’s needs. Cogent Arts & Humanities, 6(1), 1-7. https://doi.org/10.1080/23311983.2019.1601055
García, I. (2021). Post-editing in the age of neural machine translation: Challenges and opportunities. Journal of Translation Technology, 27(1), 45-60.
Gil, J. R. B., & Pym, A. (2006). Technology and translation (a pedagogical overview). Translation Technology and Its Teaching, Intercultural Studies Group, Universitat Rovira i Virgili, Tarragona.
Granger, S., & Lefer, M. A. (2023). Learner translation corpora: Bridging the gap between learner corpus research and corpus-based translation studies. International Journal of Learner Corpus Research, 9(1), 1-28.
Guerberof-Arenas, A., & Toral, A. (2023). The impact of automated feedback on translator trainees: A mixed-methods study on post-editing and cognitive load. Translation, Cognition & Behavior, 6(1), 78–102. https://doi.org/10.1075/tcb.00042.gue
Koponen, M. (2015). How to teach machine translation post-editing? Experiences from a post-editing course. In Proceedings of the 4th Workshop on Post-editing Technology and Practice.
Koponen, M., Aziz, W., & Castilho, S. (2021). A framework for post-editing quality assessment. Machine Translation Journal, 35(4), 241-259.
Koponen, M., Salmi, L., & Nikulin, M. (2020). Machine translation and post-editing: The impact of feedback on translation quality. Translation Spaces, 9(1), 123–145.
Laviosa, S. (2021). Corpus-based translation studies: Theory, findings, applications (Vol. 17). Brill.
Lommel, A. (2018). Metrics for translation quality assessment: A case for standardising error typologies. Translation quality assessment: From principles to practice, 109-127.
Mahdy, O. S. M. M. S., Samad, S. S., & Mahdi, H. S. (2020). The attitudes of professional translators and translation students towards computer-assisted translation tools in Yemen. Journal of Language and Linguistic Studies, 16(2), 1084-1095.
Martikainen, H. (2018). A functional approach to translation quality assessment: Categorizing sources of translational distortion in medical abstracts. Linguistica Antverpiensia, New Series–Themes in Translation Studies, 16, 106-121. https://doi.org/10.52034/lanstts.v16i0.435
Massey, G., & Ehrensberger-Dow, M. (2011). Investigating information literacy: A growing priority in translation studies. Across Languages and Cultures, 12(2), 193-211.
Mitchell, L., O’Brien, S., & Roturier, J. (2014). Quality evaluation in community post-editing. Machine translation, 28, 237-262.
Moorkens, J. (2022). Ethics and machine translation. In Dorothy Kenny (Ed.), Machine translation for everyone: Empowering users in the age of artificial intelligence, 121–140. Berlin: Language Science Press. DOI: 10.5281/zenodo. 6759984
Moorkens, J., Toral, A., Castilho, S., & Way, A. (2018). Translators’ perceptions of literary post-editing using statistical and neural machine translation. Translation Spaces, 7(2), 240-262. https://doi.org/10.1075/ts.18014.moo
Muñoz Andrés, P. (2024). The Impact of ISO Certifications, Machine Translation (MT), & Large Language Models (LLMs) in the Quality of English into Spanish Translations [Doctoral dissertation]. University of Salamanca. http://hdl.handle.net/10366/163499
Plyth, P. S., & Craham, C. P. (2023). Translation affects literary and cultural systems: how to observe the features of translation? Applied Translation, 17(1), 7-15.
Rodríguez-Castro, M. (2018). An integrated curricular design for computer-assisted translation tools: developing technical expertise. The Interpreter and Translator Trainer, 12(4), 355-374.
Samman, H. M. (2022). Evaluating machine translation post-editing training in undergraduate translation programs-an exploratory study in Saudi Arabia [Doctoral dissertation]. University of Southampton.
Terribile, S. (2024). Productivity in the post-editing of neural machine translation: A mixed-methods analysis of speed and edits at Toppan Digital Language [PhD Dissertation]. University of Manchester.
Toral, A., & Way, A. (2023). Post-editing neural machine translation: Assessing accuracy and efficiency. Computational Linguistics Journal, 49(2), 211-233.
Wang, X., & Wang, J. (2023). Implementing an interactive approach in translator education. Interactive Learning Environments, 31(4), 2288-2304. https://doi.org/10.1080/10494820.2021.1879871
Winer, L. R., & Cooperstock, J. (2002). The “intelligent classroom”: Changing teaching and learning with an evolving technological environment. Computers & Education, 38(1-3), 253-266.
Yamada, M. (2019). The impact of Google neural machine translation on post-editing by student translators. The Journal of Specialised Translation, 31(1), 87-106.
Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020). Applying machine learning in science assessment: A systematic review. Studies in Science Education, 56(1), 111-151.
