Exploring AI-Driven Written English Assessment: Toward Improved Assessment Quality and Learner Outcomes
DOI:
https://doi.org/10.17507/jltr.1702.26Keywords:
contextual sensitivity, discourse competence, technological efficiency, pedagogical integrity, validity and fairnessAbstract
The use of AI-powered tools for assessing written English is revolutionizing conventional evaluation methods by improving accuracy, consistency, and teaching efficiency. However, there is limited research on how AI assessment tools affect the validity and fairness of language evaluation. This review synthesizes current research on the technological foundations, pedagogical impact, and ethical considerations of AI in this area. This review assesses the role of AI in education, highlighting AI-driven technologies such as automated writing evaluation and natural language processing that provide timely feedback, support personalized learning, and reduce instructor workload. The study employed the PRISMA method to investigate the impact of AI on written English assessment by analyzing empirical studies from Scopus, Web of Science, and Google Scholar. The findings revealed that AI tools significantly enhanced the evaluation of written English by providing reliable assessments, immediate feedback, fostering learner autonomy, and producing assessments consistent with those of human raters, thereby promoting self-regulated writing and increasing access to quality feedback. However, specific concerns about algorithmic bias, feedback clarity, and the diminishing role of human judgment in assessing creativity and discourse were also found. It was also found that educational automation negatively impacts linguistic diversity, critical thinking, data privacy, transparency, and equitable access to AI tools. The review suggests integrating AI capabilities with human experience to balance technological efficiency and pedagogical integrity in the use of AI tools. It recommends developing hybrid assessment frameworks that integrate AI analytics with human evaluation to enhance fairness, accuracy, and comprehension in written English assessment.
References
Alharbi, W. (2023). AI in the Foreign Language Classroom: A Pedagogical Overview of Automated Writing Assistance Tools. Education Research International, 2023, 1–15. https://doi.org/10.1155/2023/4253331
Al-Obaydi, L. H., & Pikhart, M. (2025). Correction: AI partner versus human partner: comparing AI-based peer assessment with human-generated peer assessment in examining writing skills. Language Testing in Asia, 15(1). https://doi.org/10.1186/s40468-025-00383-8
Bachman, L. F. (2024). What Is the Construct? The Dialectic of Abilities and Contexts in Defining Constructs in Language Assessment 1. In The Writings of Lyle F. Bachman (pp. 220-250). Routledge.
Biju, N., Abdelrasheed, N. S. G., Bakiyeva, K., Prasad, K. D. V., & Jember, B. (2024). Which one? AI-assisted language assessment or paper format: an exploration of the impacts on foreign language anxiety, learning attitudes, motivation, and writing performance. Language Testing in Asia, 14(1). https://doi.org/10.1186/s40468-024-00322-z
Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability (formerly: Journal of personnel evaluation in education), 21(1), 5-31. https://doi.org/10.1007/s11092-008-9068-5
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325.
Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1). https://doi.org/10.1186/s41239-023-00411-8
Chan, M., & Luckin, R. (2020). Preparing university assessment for a world with AI: Tasks for human intelligence. In Re-imagining university assessment in a digital world (pp. 49-63). Cham: Springer International Publishing.
Chen, A., Zhang, Y., Jia, J., Liang, M., Cha, Y., & Lim, C. P. (2024). A systematic review and meta‐analysis of AI‐enabled assessment in language learning: Design, implementation, and effectiveness. Journal of Computer Assisted Learning, 41(1). https://doi.org/10.1111/jcal.13064
Cheng, Y. (2025). Transforming English language education with AI-driven deep learning models for scalable adaptive and inclusive assessment. International Journal of Information and Communication Technology, 26(12), 15–31. https://doi.org/10.1504/ijict.2025.146163
Deepshikha, D. (2025). A comprehensive review of AI-powered grading and tailored feedback in universities. Discover Artificial Intelligence, 5(1). https://doi.org/10.1007/s44163-025-00517-0
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., ... & Vayena, E. (2018). AI4People—An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and machines, 28(4), 689-707.
Ghaithi, A. A., & Behforouz, B. (2025). Effects of interaction with AI-assisted writing evaluation on EFL students’ writing performance. Knowledge Management & E-Learning, 206–224. https://doi.org/10.34105/j.kmel.2025.17.009
Hooda, M., Rana, C., Dahiya, O., Rizwan, A., & Hossain, M. S. (2022). Artificial intelligence for assessment and feedback to enhance student success in higher education. Mathematical Problems in Engineering, 2022, 1–19. https://doi.org/10.1155/2022/5215722
Jamshed, M., Fatimah Albedah, Mohd Sajid Ansari, & Sameena Banu. (2025). Assessing the Efficacy of AI-Driven Corrective Feedback via WhatsApp Application to Improve ESL Learners’ Writing Skills: An Experimental Study. International Journal of Interactive Mobile Technologies (iJIM), 19(07), 190–205. https://doi.org/10.3991/ijim.v19i07.52709
Jamshed, M., Manjur Ahmed, A. S. M., Sarfaraj, M., & Warda, W. U. (2024). The Impact of ChatGPT on English Language Learners' Writing Skills: An Assessment of AI Feedback on Mobile. International Journal of Interactive Mobile Technologies, 18(19), 18-36. https://doi.org/10.3991/ijim.v18i19.50361
Jiang, Z., Xu, Z., Pan, Z., He, J., & Xie, K. (2023). Exploring the role of artificial intelligence in facilitating assessment of writing performance in second language learning. Languages, 8(4), 247.
Jin, Y., & Fan, J. (2023). Test-Taker Engagement in AI Technology-Mediated Language Assessment. Language Assessment Quarterly, 20(4–5), 488–500. https://doi.org/10.1080/15434303.2023.2291731
Kanchana, S., & Saha, P. R. (2025). Evolving Student Assessment: AI-Driven Rubrics for Personalized and Equitable English Language Learning. Journal of Engineering Education Transformations, 38(2), 584–590. https://doi.org/10.16920/jeet/2025/v38is2/25072
Khan, M. A., Kurbonova, O., Abdullaev, D., Radie, A. H., & Basim, N. (2024). Is AI-assisted assessment liable to evaluate young learners? Parents support, teacher support, immunity, and resilience are in focus in testing vocabulary learning. Language Testing in Asia, 14(1). https://doi.org/10.1186/s40468-024-00324-x
Khasawneh, D. M. A. S. K. (2024). Improving the learning of language proficiency at tertiary education level through AI-driven assessment models and automated feedback systems. Migration Letters, 21(2), 712-726.
Lee, O. (2024). Examining AI-Based Accuracy Assessment in L2 learners’ writing. Journal of Pan-Pacific Association of Applied Linguistics, 28(2), 39–55. https://doi.org/10.25256/paal.28.2.3
Li, W. Y., Kau, K., & Shiung, Y. J. (2023). Pedagogic exploration into adapting automated writing evaluation and peer review integrated feedback into large-sized university writing classes. SAGE Open, 13(4), 1-18. https://doi.org/10.1177/215824402312090
Liao, X., Zhang, X., Wang, Z., & Luo, H. (2024). Design and implementation of an AI‐enabled visual report tool as formative assessment to promote learning achievement and self‐regulated learning: An experimental study. British Journal of Educational Technology, 55(3), 1253-1276.
Mahdi, H. S., & Alkhateeb, A. (2025). Revolutionising essay evaluation: a cutting-edge rubric for AI-assisted writing. International Journal of Computer-Assisted Language Learning and Teaching (IJCALLT), 15(1), 1-19.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741–749. https://doi.org/10.1037/0003-066x.50.9.741
Poonpon, K., Manorom, P., & Chansanam, W. (2023). Exploring effective methods for automated essay scoring of non-native speakers. Contemporary Educational Technology, 15(4), ep475. https://doi.org/10.30935/cedtech/13740
Rahman, N. a. A., Zulkornain, L. H., Mat, A. C., & Kustati, M. (2023). Assessing Writing Abilities using AI-Powered Writing Evaluations. Journal of ASIAN Behavioural Studies, 8(24), 1–17. https://doi.org/10.21834/jabs.v8i24.420
Ranalli, J. (2018). Automated written corrective feedback: how well can students make use of it? Computer Assisted Language Learning, 31(7), 653–674. https://doi.org/10.1080/09588221.2018.1428994
Shi, H., & Aryadoust, V. (2024). A systematic review of AI-based automated written feedback research. ReCALL, 36(2), 187–209. https://doi.org/10.1017/s0958344023000265
Sweller, J. (2011). Cognitive load theory. In Psychology of learning and motivation (Vol. 55, pp. 37-76). Academic Press.
Tam, A. C. F. (2024). Interacting with ChatGPT for internal feedback and factors affecting feedback quality. Assessment & Evaluation in Higher Education, 50(2), 219–235. https://doi.org/10.1080/02602938.2024.2374485
Tapalova, O., & Zhiyenbayeva, N. (2022). Artificial Intelligence in Education: AIED for Personalised Learning Pathways. The Electronic Journal of e-Learning, 20(5), 639–653. https://doi.org/10.34190/ejel.20.5.2597
Usher, M. (2025). Generative AI vs. instructor vs. peer assessments: a comparison of grading and feedback in higher education. Assessment & Evaluation in Higher Education, 50(6), 912–927. https://doi.org/10.1080/02602938.2025.2487495
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes (Vol. 86). Harvard University Press.
Wang, Z. (2020). Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course. Library Hi Tech, 40(1), 80–97. https://doi.org/10.1108/lht-05-2020-0113
Weigle, S. C. (2002). Assessing writing. Cambridge University Press.
Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P. L. H. (2022). AI-assisted automated scoring of picture-cued writing tasks for language assessment. Education and Information Technologies, 28(6), 7031–7063. https://doi.org/10.1007/s10639-022-11473-y
Zimmerman, B. J. (2002). Becoming a Self-Regulated Learner: An Overview. Theory into Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102_2
