Establishing the Validity and Reliability of the Malaysian English Language Textbook Evaluation Checklist (MELTEC) Using Rasch Measurement Model (RRM)
DOI:
https://doi.org/10.17507/jltr.1301.05Keywords:
validity, reliability, textbook evaluation checklist, Rasch Measurement Model (RRM)Abstract
A textbook is an important teaching and learning material. Evaluating a textbook is essential to ensure educational goals are met. In the present study, the Malaysian English Language Textbook Evaluation Checklist (MELTEC) is developed for the purpose of materials adaptation. The lack of validity and reliability of previous textbook evaluation checklists reported by researchers in the field of ELT textbook evaluation has resulted in the need to establish the validity and reliability of the checklist developed in this study using the Rasch Measurement Model (RRM). One hundred and ten ESL secondary school teachers from Kuala Lumpur, Malaysia, had participated in this study. Rasch analysis was used to examine the item fit, polarity, reliability, separation index, and unidimensionality of the checklist. As a result, the finalised version of MELTEC comprises of 78 items. A total of 63 items were dropped as these items were not deemed to be suitable. The analysis performed using Rasch Measurement Model proved that the checklist has high validity and reliability. The checklist is found to be suitable to evaluate ELT secondary school textbooks in Malaysia. On that account, future researchers can consider using Rasch Measurement Model to establish the validity and reliability of the instruments developed to ensure their integrity and quality.
References
AbdelWahab, M. M. (2013). Developing an English Language Textbook Evaluative Checklist. Journal of Research & Method in Education, 1(3), 55-70.
Aryadoust, V., Tan, H. A. H., & Ng, L. Y. (2019). A Scientometric Review of Rasch Measurement: The Rise and Progress of a Specialty. Frontiers in Psychology, 10(2197), 1–16.
Bond, T. G., & Fox, C. M. B. (2015). Applying the Rasch: Fundamental measurement in the Human Sciences (Third Edition). New York: Routledge.
Cunningsworth, A. (1995). Choosing Your Coursebook (Handbooks for the English Classroom). Oxford: Macmillan Heinemann.
Demir, Y., & Ertas, A. (2014). A suggested eclectic checklist for ELT coursebook evaluation. The Reading Matrix, 14(2), 243–252.
Fisher, W. P. (2007). Rating scale instrument quality criteria. Rasch Measurement Transactions, 21(1), 1095.
Fox, C. M., & Jones, J. A. (1998). Uses of Rasch Modeling in Counseling Psychology Research. Journal of Counseling Psychology, 45(1), 30–45.
Halim, S., & Halim, T. (2016). Adapting Materials: Revisiting the Needs of Learners. International Journal of Humanities and Cultural Studies, 2(4), 633–642.
Hassan, A. B. (2012). Instrumen Penilaian Pembimbing Dalam Pelaksanaan Pembelajaran Berasaskan Kerja Pelajar Di Industri. [Unpublished Ph.D. dissertation], Universiti Teknologi Malaysia.
Işik, A. (2018). ELT Materials Evaluation : A System and Criteria. Theory and Practice in Language Studies, 8(7), 797–812.
Jamil, M. R. B. M., Siraj, S. B., Hussin, Z. B., Noh, N. R. B. M., & Sapar, A. A. B. (2019). Pengenalan Asas Kaedah Fuzzy Delphi: Dalam Penyelidikan Reka Bentuk Pembangunan (Third Edit). Minda Intelek Agency.
Kamis, A., Bakar, A. R., Hamzah, R., Soaib A., & Zaliza, H. (2015). Using Rasch model and confirmatory factor analysis to assess instrument for clothing fashion design competency. International Journal of Social Science and Humanity, 5(5), 418–421.
Karamifar, M., Barati, H., & Youhanaee, M. (2014). Local Evaluation Criteria for Global Textbooks: A Case Study from Iran. Theory and Practice in Language Studies, 4(5), 923–930.
Linacre, J. (2002). Understanding Rasch measurement: Optimizing Rating Scale Category Effectiveness. Journal of Applied Measurement, 3, 85–106.
Lopez Medina, B. (2016). Developing a CLIL textbook evaluation checklist. Latin American Journal of Content & Language Integrated Learning, 9(1), 159–173.
Maat, S. M., Effendi, M. E., Affandi, H. M., & Mohamed, S. (2018). Psychometric evidence of statistical self-efficacy instrument based on postgraduate. International Journal of Advanced and Applied Sciences, 5(11), 95–99.
McDonough, J., Shaw, C., & Masuhara, H. (2013). Materials and Methods in ELT: A Teacher’s Guide (Third Edit). West Sussex: Wiley-Blackwell.
McGrath, I. (2002). Materials Evaluation and Design for Language Teaching. Edinburgh: Edinburgh University Press.
Mukundan, J., & Ahour, T. (2010). A review of Textbook Evaluation Checklist across Four Decades (1970-2008). In Tomlinson, B. & Mahasura, H. (Eds.). Research for Materials Development in Language Learning: Evidence for best practice (pp. 336–352). Continuum International Publishing.
Nehal, R. (2016). Materials Adaptation in English Language Teaching : Implications for Teacher Development. In Rahman, M. M. (Ed.) (2016). Integrating Technology and Culture: Strategies and Innovations in ELT (pp. 239–262), Anubhav Publishing House: India.
Nguyen, C. (2015). An Evaluation of the Textbook English 6 : A case study from secondary schools in the Mekong Delta Provinces of Vietnam. [Unpublished Ph.D. dissertation], University of Sheffield.
Nimehchisalem, V., & Mukundan, J. (2013). Usefulness of the English Language Teaching Textbook Evaluation Checklist. Pertanika Journal of Social Sciences and Humanities, 21(2), 797–816.
Nimehchisalem, V., & Mukundan, J. (2015). Refinement of the English Language Teaching Textbook Evaluation Checklist. Pertanika Journal of Social Sciences and Humanities, 23(4), 761–780.
Reckase, M. D. (1979). Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications. Journal of Educational and Behavioral Statistics, 4(3), 207–230.
Sahin, S. (2020). Developing a Checklist for English Language Teaching Course book Analysis. International Journal of Education and Research, 8(1), 107–120.
Simsek, M., & Dündar, E. (2018). Particularised Checklists in Materials Evaluation: Developing Contextually Relevant Criteria for Turkish EFL Classes. Journal of Language and Linguistic Studies, 14(3), 154–189.
Tomlinson, B. (2012). Materials development for language learning and teaching. Language Teaching, 45(2), 143–179.
Wright, B. D., & Stone, M. H. (1979). Best Test Design. MESA Press: Chicago, IL
Zokaeieh, S., Karimi, S. N., Nouri, F., & Hakimzadeh, A. (2019). Developing a Critical Checklist for Textbook Evaluation. The Journal of English Language Pedagogy and Practice, 12(24), 202–225.
