Comparing English Language Learners’ Perceptions of How Reliable Computer-Based, Teacher-Based, and Peer Feedback Is: A Case Study
DOI:
https://doi.org/10.17507/jltr.1303.24Keywords:
feedback, automated systems, reliability, academic writing, perceptionsAbstract
The literature emphasises the role of feedback (FB) in writing development, leading to explorations of different types of FB to provide, such as teacher-based FB (TBF), peer feedback (PF) and computer-based feedback (CBF). This quasi-experimental study aimed to investigate EFL learners’ perceptions of the reliability of TBF, PF and CBF. The participants (n = 40) were Saudi male EFL students in a BA English programme at a Saudi university. The study employed an experimental group (n = 21) and a control group (n = 19). For data collection, pre- and post-intervention questionnaires were administered. The intervention exposed the participants to giving and receiving PF and introduced them to CBF. Following training in providing PF and using the automated system, the participants went through four cycles of writing during which they developed four essays; with each essay, PF and CBF were employed to produce multiple drafts. The main findings indicated that TBF was perceived to be the most reliable type of FB, and that CBF was considered more reliable than PF. Additionally, our findings suggest that the more students are exposed to CBF, the more likely they are to accept it. Pedagogical implications arising from these findings are also discussed.
References
Alnasser, S. (2013). A New Form of Peer Feedback Technique: An Investigation into the Impact of Focusing Saudi ESL Learners on Macro Level Writing Features. Ph.D. Thesis. Newcastle University.
Alnasser, S. M. N. (2018). Exploring Student-Writers’ Views on Replacing Teacher Feedback with Peer Feedback and Computer-Based Feedback. Arab World English Journal, 9(3), 345-366.
Alzahrani, M., & O’Toole, J. (2017). The Impact of Internet Experience and Attitude on Student Preference for Blended Learning. Journal of Curriculum and Teaching, 6(1), 65-78. https://doi.org/10.5430/jct.v6n1p65
Bitchener, J., & Ferris, D. (2012). Written Corrective Feedback in Second Language Acquisition and Writing. New York: Routledge.
Brown, W., Lovett, M., Bajzek, D., & Burnette, J. (2006). Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard. HI, USA.
Burkhart, C., Lachner, A., & Nückles, M. (2020). Assisting Students’ Writing with Computer-Based Concept Map Feedback: A Validation Study of the CohViz Feedback System. PLOS ONE, 15(6), e0235209. https://doi.org/10.1371/journal.pone.0235209
Chang, Y., Lee, C., Chen, C., Liao, C., Ng, C., Chen, J., & Chaou, C. (2017). Exploring the Influence of Gender, Seniority and Specialty on Paper and Computer-Based Feedback Provision During Mini-CEX Assessments in a Busy Emergency Department. Advances in Health Sciences Education, 22(1), 57-67. DOI: 10.1007/s10459-016-9682-9
Chang, H., & Windeatt, S. (2021). Designing and Applying a Moodle-Based E-Textbook for an Academic Writing Course. International Journal of Mobile and Blended Learning, 13(2), 73-95.
Chen, J., White, S., McCloskey, M., Soroui, J., & Chun, Y. (2011). Effects of Computer Versus Paper Administration of an Adult Functional Writing Assessment. Assessing Writing, 16(1), 49-71. https://doi.org/10.1016/j.asw.2010.11.001
Cheung, A., & Slavin, R. (2012). How Features of Educational Technology Applications Affect Student Reading Outcomes: A Meta-Analysis. Educational Research Review, 7(3), 198-215. https://doi.org/10.1016/j.edurev.2012.05.002
Cho, K., & Schunn, C. (2007). Scaffolded Writing and Rewriting in the Discipline: A Web-Based Reciprocal Peer Review System. Computers and Education, 48(3), 409-426. https://doi.org/10.1016/j.compedu.2005.02.004
Coniam, D. (2009). Experimenting with a Computer Essay-Scoring Program Based on ESL Student Writing Scripts. ReCALL, 21(2), 259-279. https://doi.org/10.1017/S0958344009000147
Cohen, A., & Cavalcanti, M. (1990). Feedback on Compositions: Teacher and Student Verbal Reports. Cambridge: Cambridge University Press.
Dikli, S. (2006). An Overview of Automated Scoring of Essays. Journal of Technology, Learning, and Assessment, 5(1), 1-36.
Deane, P., Quinlan, T., & Kostin, I. (2011). Automated Scoring Within a Developmental, Cognitive Model of Writing Proficiency. ETS Research Report Series, 1, i-93. doi: 10.1002/j.2333-8504.2011.tb02252.x.
El Ebyary, K., & Windeatt, S. (2010). The Impact of Computer-Based Feedback on Students’ Written Work. International Journal of English Studies, 10(2), 121-142. https://doi.org/10.6018/ijes/2010/2/119231
El Ebyary, K., & Windeatt, S. (2017). Eye Tracking Analysis of EAP Student’s Regions of Interest in Computer-Based Feedback on Grammar, Usage, Mechanics, Style and Organization and Development. Occasional Papers in the Development of English Education, 63(1), 5-30. DOI: 10.21608/OPDE.2017.87705
Ernst B. & Steinhauser M. (2018). Effects of Feedback Reliability on feedback-Related Brain Activity: A feedback valuation Account. Cognitive, Affective, & Behavioural Neuroscience, 18, 596-608. https://doi.org/10.3758/s13415-018-0591-7
Ferris, D. (1995). Student Reactions to Teacher Response in Multiple-Draft Composition Classrooms. TESOL Quarterly, 29(1), 33-53. https://doi.org/10.2307/3587804
Ferris, D., & Roberts, B. (2001). Error Feedback in L2 Writing Classes? Journal of Second Language Writing, 10(3), 161-184. https://doi.org/10.1016/S1060-3743(01)00039-X
Gibbs, G., & Simpson, C. (2002). How Assessment Influences Student Learning: A Conceptual Overview. CHEP: The Open University.
Gibbs, G., & Simpson, C. (2004). Conditions Under Which Assessment Supports Student Learning. Learning and Teaching in Higher Education, 1(1), 3-31.
Gomez, R., Parker, R., Lara-Alecio, R., & Gomez, L. (1996). Process Versus Product Writing with Limited English Proficient Students. Bilingual Research Journal, 20(2), 209-233. https://doi.org/10.1080/15235882.1996.10668628
Grabe, W., & Kaplan, R. P. (1996). Theory and Practice of Writing: An Applied Linguistic Perspective. London: Longman.
Grimes, D., & Warschauer, M. (2010). Utility in a Fallible Tool: A Multi-Site Case Study of Automated Writing Evaluation. Journal of Technology, Learning, and Assessment, 8(6), 1-43.
Haigh, M. (2007). Sustaining Learning Through Assessment: An Evaluation of the Value of a Weekly Class Quiz. Assessment and Evaluation in Higher Education, 32(4), 457-474. https://doi.org/10.1080/02602930600898593
Helfaya, A. (2019). Assessing the Use of Computer-Based Assessment-Feedback in Teaching Digital Accountants. Accounting Education, 28(1), 69-99. https://doi.org/10.1080/09639284.2018.1501716
Hoq, M. (2020). E-Learning During the Period of Pandemic (COVID-19) in the Kingdom of Saudi Arabia: An Empirical Study. American Journal of Educational Research, 8(7), 457-464. doi:10.12691/education-8-7-2.
Hu, G. (2005). Using Peer Review with Chinese ESL Student Writers. Language Teaching Research, 9(3), 321-342. https://doi.org/10.1191/1362168805lr169oa
Huxham, M. (2007). Fast and Effective Feedback: Are Model Answers the Answer? Assessment and Evaluation in Higher Education, 32(6), 601-611. https://doi.org/10.1080/02602930601116946
Hyland, K. (2003). Second Language Writing. Cambridge: Cambridge University Press.
Hyland, K., & Hyland, F. (2006). Interpersonal Aspects of Response: Constructing and Interpreting Teacher Written Feedback. Cambridge: Cambridge University Press.
Jarom, E., Woodruff, E., Bryson, M., & Lindsay, P. (1991). The Effects of Revising with a Word Processor on Writing Composition. Research in the Teaching of English, 26, 167-193.
Lachner, A., & Neuburg, C. (2019). Learning by Writing Explanations: Computer-Based Feedback About the Explanatory Cohesion Enhances Students’ Transfer. Instructional Science, 47(1), 19-37. https://doi.org/10.1007/s11251-018-9470-4
Latifi, S., Noroozi, O., Hatami, J. & Biemans, H. (2021). How Does Online Peer Feedback Improve Argumentative Essay Writing and Learning? Innovations in Education and Teaching International. 58(2), 195-206. https://doi.org/10.1080/14703297.2019.1687005
Lee, I. (2007). Feedback in Hong Kong Secondary Writing Classrooms: Assessment for Learning or Assessment of Learning? Assessing Writing, 12(3), 180-198. https://doi.org/10.1016/j.asw.2008.02.003
Leki, I. (1990). Potential Problems with Peer Responding in ESL Writing Classes. CATESOL Journal, 3, 5-19.
Li, J. (2006). The Mediation of Technology in ESL Writing and Its Implications for Writing Assessment. Assessing Writing, 11(1), 5-21. https://doi.org/10.1016/j.asw.2005.09.001
Li, Z. (2021). Teachers in Automated Writing Evaluation (AWE) System-Supported ESL writing classes: Perception, implementation, and influence. System, 99, 1–14. https://doi.org/10.1016/j.system.2021.102505
Lizzio, A., & Wilson, K. (2008). Feedback on Assessment: Students’ Perceptions of Quality and Effectiveness. Assessment and Evaluation in Higher Education, 33(3), 263-275. https://doi.org/10.1080/02602930701292548
McCutchen, D. (2011). From Novice to Expert: Implications of Language Skills and Writing Relevant Knowledge for Memory during the Development of Writing Skill. Journal of Writing Research, 3(1), 51-68. http://dx.doi.org/10.17239/jowr-2011.03.01.3
Miller, T. (2009). Formative Computer-Based Assessment in Higher Education: The Effectiveness of Feedback in Supporting Student Learning. Assessment and Evaluation in Higher Education, 34(2), 181-192. https://doi.org/10.1080/02602930801956075
Min, H. (2008). Reviewer Stances and Writer Perceptions in EFL Peer Review Training. English for Specific Purposes, 27(3), 285-305. https://doi.org/10.1016/j.esp.2008.02.002
Mohamed, A. (2008). Effects of Active Learning Variants on Student Performance and Learning Perceptions. International Journal for the Scholarship of Teaching and Learning, 2(2), 1-15. https://doi.org/10.20429/ijsotl.2008.020211
Morgan, H. (2020). Best Practices for Implementing Remote Learning During a Pandemic. The Clearing House, 93(3), 135-141. https://doi.org/10.1080/00098655.2020.1751480
OECD (2005). Formative Assessment: Improving Learning in Secondary Classrooms. Centre for Educational Research and Innovation. Paris: OECD Publication Paris.
Peterson, E., & Irving, S. (2008). Secondary School Students’ Conceptions of Assessment and Feedback. Learning and Instruction, 18(3), 238-250. https://doi.org/10.1016/j.learninstruc.2007.05.001
Powers, D., Burstein, J., Chodorow, M., Fowles, M., & Kukich, K. (2001). Stumping e-Rater: Challenging the Validity of Automated Essay Scoring. Computers in Human Behavior, 18(2), 103-134. https://doi.org/10.1016/S0747-5632(01)00052-8
Phuwichit, K. (2016). A Study of Teacher Feedback on Peer Feedback in EFL Writing and Its Relation to Self-Regulation. PhD Thesis. University of Southampton.
Reid, J. (1997). Responding to ESL Student Language Problems: Error Analysis and Revision Plans. Boston: Heinle & Heinle.
Rollinson, P. (2005). Using Peer Feedback in the ESL Writing Class. ELT Journal, 59(1), 23-30. https://doi.org/10.1093/elt/cci003
Rudner, L., & Liang, T. (2002). Automated Essay Scoring Using Bayes’ Theorem. Journal of Technology, Learning, and Assessment, 1(2), 1-22.
Tang, G., & Thitecott, J. (1999). Peer Response in ESL Writing. TESL Canada Journal, 16(2), 20-38. https://doi.org/10.18806/tesl.v16i2.716
Tanveer, M., Bhaumik, A., Hassan, S., & U Haq, I. (2020). Covid-19 Pandemic, Outbreak Educational Sector and Students Online Learning in Saudi Arabia. Journal of Entrepreneurship Education, 23(3), 1-14.
Truscott, J., & Hsu, A. (2008). Error Correction, Revision, and Learning. Journal of Second Language Writing, 17(4), 292-305. https://doi.org/10.1016/j.jslw.2008.05.003
Van Steendam, E., Rijlaarsdam, G., Sercu, L., & Van den Bergh, H. (2010). The Effect of Instruction Type and Dyadic or Individual Emulation on the Quality of Higher-Order Peer Feedback in EFL. Learning and Instruction, 20(4), 316-327. https://doi.org/10.1016/j.learninstruc.2009.08.009
Vygotsky, L. (1978). Mind in Society: Development of Higher Psychological Processes. USA: Harvard University Press.
Walker, S., & Patel, A. (2018). More than Skills: What Can Approaches to Digital Literacies Learn from Academic Literacies? Journal of Learning and Teaching in Higher Education, 1(1), 93-100.
Wang, J., & Brown, M. (2007). Automated Essay Scoring Versus Human Scoring: A Comparative Study. Journal of Technology, Learning, and Assessment, 6(2), 1-29.
Wei, H., & Chou, C. (2019). Relationships Among College Learners’ Online Learning Perceptions, Behaviors, and Achievements via the Self-Determination Theory Approach. Toronto: AERA Annual Meeting Proceedings.
Wei, H., & Chou, C. (2020). Online Learning Performance and Satisfaction: Do Perceptions and Readiness Matter? Distance Education, 41(1), 48-69. https://doi.org/10.1080/01587919.2020.1724768
Williams, J. (2002). Undergraduate Second Language Writers in The Writing Center. Journal of Basic Writing, 21 (2), pp. 73-91. https://www.jstor.org/stable/43744176, DOI: 10.37514/JBW-J.2002.21.2.06.
Yang, M., Badger, R., & Yu, Z. (2006). A Comparative Study of Peer and Teacher Feedback in a Chinese EFL Writing Class. Journal of Second Language Writing, 15(3), 179-200. https://doi.org/10.1016/j.jslw.2006.09.004
Yu, S. (2021). Giving Genre-Based Peer Feedback in Academic Writing: Sources of Difficulties and Challenges. Assessment & Evaluation in Higher Education, 46(1), 36-53. https://doi.org/10.1080/02602938.2020.1742872
Zaini, A., & Mazdayasna, G. (2015). The Impact of Computer-Based Instruction on the Development of EFL Learners’ Writing Skills. Journal of Computer Assisted Learning, 31(6), 516-528. https://doi.org/10.1111/jcal.12100
