Does Translation Technology Affect Translators' Performance? A Meta-Analysis
DOI:
https://doi.org/10.17507/jltr.1505.35Keywords:
CAT tools, meta-analysis, translator’s experience, translation technologies, translators’ performanceAbstract
Translation technologies, including computer-assisted translation (CAT) tools, translation memory (TM) systems, and machine translation (MT), are increasingly utilized in professional translation workflows and training. However, the effects of these technologies on translators' performance remain inconclusive. This meta-analysis examines the overall impact of translation technologies on translator performance by synthesizing data from 12 experimental studies published between 2000 and 2023. The study investigates the effectiveness of translation technologies compared to traditional translation methods. The findings reveal a significant positive effect size, indicating that translation technologies have the potential to improve translators’ performance relative to purely human translation. The integration of advanced interactive CAT systems and post-editing MT demonstrates larger advancements compared to basic TM match retrieval. Moreover, experienced professional translators derive greater benefits from incorporating technologies than student translators, highlighting the importance of leveraging automation capabilities alongside human expertise. However, the study identifies significant heterogeneity among the studies, influenced by factors such as translation direction. Translators translating into their native language exhibit greater advancements, emphasizing the advantages of technologies that strengthen fluency in the target language.
References
Alotaibi, H. M. (2017). Arabic-English Parallel Corpus: A New Resource for Translation Training and Language Teaching. Arab World English Journal, 8(3), 319 –337. DOI: https://dx.doi.org/10.24093/awej/vol8no3.21
Alotaibi, H. M. (2020). Computer-Assisted Translation Tools: An Evaluation of Their Usability among Arab Translators. Applied Sciences, 10(18), 6295. https://doi.org/10.3390/app10186295
Alotaibi, H., and Salamah, D. (2023). The impact of translation apps on translation students’ performance. Education and Information Technologies, 28(8), 10709–10729. https://doi.org/10.1007/s10639-023-11578-y
Borenstein, M. (2012). Effect size estimation. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, and K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol 3: Data analysis and research publication. (pp. 131–146). American Psychological Association. https://doi.org/10.1037/13621-006
Borenstein, M. and Rothstein, H. (1999). Comprehensive Meta-Analysis: A Computer Program for Research Synthesis. Version 1.0. 23 [Computer Software], Biostat, Englewood Cliffs.
Carl, M., Dragsted, B., Elming, J., Hardt, D., and Jakobsen, A. L. (2011). The process of post-editing: A pilot study. Copenhagen Studies in Language, 41(1), 131-142.
Cohen, J. (1988). The effect size. Statistical Power Analysis for the Behavioral Sciences, 77–83.
Cohn, L. D., and Becker, B. J. (2003). How meta-analysis increases statistical power. Psychological Methods, 8(3), 243–253. https://doi.org/10.1037/1082-989X.8.3.243
Dehbashi Sharif, F., and Salehi, F. (2015). The Effect of the Translation Technologies on the Critical Thinking of the Students of the Translation Studies. Journal of Language and Translation, 5(1), 95-109.
Garcia, I. (2010). Is machine translation ready yet? Target. International Journal of Translation Studies, 22(1), 7–21. https://doi.org/10.1075/target.22.1.02gar
Garcia, I. (2017). Translating in the Cloud Age: Online Marketplaces. HERMES - Journal of Language and Communication in Business, 56, 59–70. https://doi.org/10.7146/hjlcb.v0i56.97202
Gaspari, F., Almaghout, H., and Doherty, S. (2015). A survey of machine translation competences: Insights for translation technology educators and practitioners. Perspectives, 23(3), 333–358. https://doi.org/10.1080/0907676X.2014.979842
Gaspari, F., and Hutchins, W. J. (2007). Online and free! Ten years of online machine translation: Origins, developments, current use and future prospects. In Proceedings of Machine Translation Summit XI (pp. 199–206). Copenhagen, Denmark: European Association for Machine Translation.
Green, S., Heer, J., and Manning, C. D. (2013). The efficacy of human post-editing for language translation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 439–448. https://doi.org/10.1145/2470654.2470718
Guerberof Arenas, A. (2013). What do professional translators think about post-editing. JoSTrans The Journal of Specialised Translation, 19, 75–95.
Hazbavi, A. A. (2011). Investigating the effect of translation memory on English into Persian translation. World Applied Sciences Journal, 15(5), 683–689.
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., and Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11(2), 193–206. https://doi.org/10.1037/1082-989X.11.2.193
Kassem, M. A. M. (2021). The effect of utilizing CAT technology on English majors’ translation and motivation. Asian EFL J, 28(2.3), 135–155.
Kong, L. (2022). Artificial Intelligence-Based Translation Technology in Translation Teaching. Computational Intelligence and Neuroscience, 2022, 6016752. https://doi.org/10.1155/2022/6016752
Lagoudaki, E. (2006). Translation Memories Survey 2006: Users’ Perceptions Around TM Use, in. Translating and the Computer 28, London, pp. 1–29.
Lagoudaki, P. M. (2009). Expanding the possibilities of translation memory systems: from the translators' wishlist to the developers' design (Doctoral dissertation, Imperial College London).
Lee, J., and Liao, P. (2011). A Comparative Study of Human Translation and Machine Translation with Post-editing. Compilation and Translation Review, 4(2), 105-149.
Melby, A. K. (2012). Terminology in the age of multilingual corpora. The Journal of Specialised Translation, 18, 7–29.
Moorkens, J., O’Brien, S., Da Silva, I. A. L., De Lima Fonseca, N. B., and Alves, F. (2015). Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 29(3–4), 267–284. https://doi.org/10.1007/s10590-015-9175-2
Morin, K. H., Barbin, F., Moreau, F., Toudic, D., and Phuez-Favris, G. (2017). Translation technology and learner performance. Translation in Transition: Between Cognition, Computing and Technology, 133, 208–234.
O’Brien, S., Ehrensberger-Dow, M., Hasler, M., and Connolly, M. (2017). Irritating CAT tool features that matter to translators. Hermes: Journal of Language and Communication in Business, 56, 145–162.
Omar, A., F., A., and El-Nabawi, I. (2020). The Impact of Translation Software on Improving the Performance of Translation Majors. International Journal of Advanced Computer Science and Applications, 11(4), 287-292 https://doi.org/10.14569/IJACSA.2020.0110439
Oswald, F. L., and Plonsky, L. (2010). Meta-analysis in Second Language Research: Choices and Challenges. Annual Review of Applied Linguistics, 30, 85–110. https://doi.org/10.1017/S0267190510000115
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., & Mulrow, C. D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. PLoS Med, 18(3), e1003583.
Plonsky, L., and Oswald, F. L. (2011). How to do a meta‐analysis. Research Methods in Second Language Acquisition: A Practical Guide, 275–295.
Plonsky, L., and Oswald, F. L. (2014). How Big Is “Big”? Interpreting Effect Sizes in L2 Research. Language Learning, 64(4), 878–912. https://doi.org/10.1111/lang.12079
Sabzalipour, B., and Rahimy, R. (2012). The Effect Of Using Supplementary On-line corpora on Advanced Iranian EFL Learners’ Translation Ability. Asian Journal of Social Sciences and Humanities, 1(3), 120-126. https://ndlsearch.ndl.go.jp/books/R000000004-I023946896
Verplaetse, H., and Lambrechts, A. (2019). Surveying the use of CAT tools, terminology management systems and corpora among professional translators: General state of the art and adoption of corpus support by translator profile1. Parallèles, 31(2), 1-31. https://doi.org/ 10.17462/para.2019.02.01
Wang, Y., Chen, L., and Han, J. (2024). Exploring factors influencing students’ willingness to use translation technology. Education and Information Technologies, 1-22. https://doi.org/10.1007/s10639-024-12511-7
