Is ChatGPT a Cure all? Demystifying the Impact of Using ChatGPT on EFL Learners’ Writing Skill

Document Type : Original Article

Authors

1 M.A. Student in Applied Linguistics, Department of ELT, Kermanshah Branch, Islamic Azad University, Kermanshah, Iran

2 Assistant Professor of Applied Linguistics, Department of ELT, Kermanshah Branch, Islamic Azad University, Kermanshah, Iran

Abstract

The aim of the current study was to demystify the impact of using ChatGPT on English as a foreign language (EFL) learners’ writing skill. To this end, 45 EFL male and female students from two language schools participated in this study, selected via convenience sampling from 67 learners based on a placement test. The participants were divided into three groups. In the first experimental group the students practiced writing using ChatGPT, while in the second experimental group, they received writing instructions using ChatGPT and teacher input, and in the third, the participants followed the conventional method was utilized. Before and after the treatment, the participants were subjected to pre and posttest. Finally, the performance of the three groups was compared via one-way ANOVA. In addition, 10 EFL learners from both experimental groups were selected voluntarily to take part in the interview and the qualitative data were analyzed based on the theme-based analysis. The findings revealed that the second experimental group outperformed the first experimental and the control groups, revealing the efficiency of the instruction via ChatGPT along with teacher input. The result of the interview showed that while the EFL students generally hold a favorable view of ChatGPT, they expressed concerns about the use of the artificial intelligence (AI) tool. In addition, they lack the necessary skills to effectively employ it to help them write. This deficiency could be a significant obstacle to the implementation of technology in their EFL writing.

Keywords

References

Abdullayeva, M., & Musayeva, Z. M. (2023). The impact of chat Gpt on students’ writing skills: an exploration of AI-assisted writing tools. International Conference of Educational Research and Innovation. 4, 61–66.
Ahmed, P.H. (2019). Major writing challenges experienced by EFL learners in Soran University. Journal of University of Human Development,5(3), 120-126. https://doi.org/10.21928/juhd.v5n3y2019.pp120-126
Alenizi, M. A. K., Mohamed, Amr M., Shaaban, T. S. (2023). Revolutionizing EFL special education: how ChatGPT is transforming the way teachers approach language learning. Innoeduca. International Journal of Technology and Educational Innovation,9(2), pp. 5-23.
Al-Jarf, R. S. (2004). Effect of online learning on struggling ESL college writers. In: NECC 2002: National Educational Computing Conference Proceedings (23rd, San Antonio, Texas, June 17-19, 2002).
Alkamel, M.A.A., & Alwagieh, N.A.S. (2024). Utilizing an adaptable artificial intelligence writing tool (ChatGPT) to enhance academic writing skills among Yemeni university EFL students. Social Sciences & Humanities,10. https://doi.org/10.1016/j.ssaho.2024.101095
Al-Mansour, N. S., & Al-Shorman, R. A. (2012). The Effect of Computer Assisted-Instruction on Saudi University Students’ Learning of English. Journal of King Saud University–Languages and Translation, 24(1), 51-56. http://dx.doi.org/10.1016/j.jksult.2009.10.001
Ausat, A., Massang, B., Efendi, M., Nofirman, N., & Riady, Y. (2023). Can Chat GPT Replace the Role of the Teacher in the Classroom: A Fundamental Analysis. Journal on Education, 5(4), 16100-16106. https://doi.org/10.31004/joe.v5i4.2745
Barrot, J. S. (2023). Using ChatGPT for second language writing: pitfalls and potentials. Assessing Writing, 5(7), 67-88:100745. https://doi.org/10.1016/j.asw.2023.100745
Beatty, K. (2003). Teaching and researching computer-assisted language learning. Longman.
Boyatzis, R.  (1998).  Transforming qualitative information:  Thematic analysis and code development.  Sage Publications.
Burston, A. R. (2021). Exploring the potential of using an AI language model for automated essay scoring. Research Methods in Applied Linguistics, 2(2), 44-59. https://doi.org/10.1016/j.rmal.2023.100050
Cao, S., Zhou, S., Luo, Y., Wang, T., Zhou, T., & Xu, Y. (2022). A review of the ESL/EFL learners’ gains from online peer feedback on English writing. Frontiers in Psychology, 13, 1035803. https://doi.org/10.3389/fpsyg.2022.1035803 
Chen, X., Zou, D., Xie, H., Cheng, G., & Liu, C. (2022). Two Decades of Artificial Intelligence in Education: Contributors, Collabo-rations, Research Topics, Challenges, and Future Directions.  Educational Technology & Society, 25(1), 28–47.
Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Sage.
Dai, K., & Liu, Q. (2024). Leveraging artificial intelligence (AI) in English as a foreign language (EFL) classes: Challenges and opportunities in the spotlight, Computers in Human Behavior, 159. https://doi.org/10.1016/j.chb.2024.108354
Dashtestani, R., & Hojatpanah, S (2020): Digital literacy of EFL students in a junior high school in Iran: voices of teachers, students and Ministry Directors. Computer Assisted Language Learning, https://doi.org/10.1080/09588221.2020.1744664
Dizon, G., & Gayed, J. (2021). Examining the impact of Grammarly on the quality of mobile L2 writing. The JALT CALL Journal, 17(2), 74–92. https://doi.org/10.29140/jaltcall.v17n2.336
Dwivedi, Y., j Sharma, A., Rana, N.P., Giannakis, M., Goel, P., Dutot, V. (2023). Evolution of artificial intelligence research in Technological Forecasting and Social Change: Research topics, trends, and future directions. Technological Forecasting and Social Change, 192. https://doi.org/10.1016/j.techfore.2023.122579
Estriegana, R. Medina, A. J. & Barchino, R.P. (2019). Student acceptance of virtual laboratory and practical work: An extension of the technology acceptance model. Computers & Education, 3(1), 33-48. https://doi.org/10.1016/j.compedu.2019.02.010
Farahian, M., & Ebadi, S. (2023). Collaborative digital writing and metacognitive knowledge in writing among TEFL students: the mediating role of online knowledge sharing. Research and Practice in Technology Enhanced Learning, 18(5), https://doi.org/10.58459/rptel.2023.18005
Fitria, T. N. (2021). Grammarly as AI-powered English writing assistant: Students’ alternative for writing English. Metathesis: Journal of English Language, Literature, and Teaching, 5(1). https://doi.org/10.31002/metathesis.v5i1.351
Golonka, E. M., Bowles, A. R., Frank, V. M., Richardson, D. L., & Freynik, S. (2014). Technologies for foreign language learning:  A review of technology types and their effectiveness. Computer     Assisted     Language Learning, 27(1), 70-105. https://doi.org/10.1080/09588221.2012.700315
Graham, F. (2022). Daily briefing: Will ChatGPT kill the essay assignment? Nature. https://doi.org/10.1038/d41586-022-04437-2
Grammarly. (2020). About Grammarly. Retrieved from https://support.grammarly.com/hc/en-   us/categories/115000018611-About- Grammarly
Hakiki, M., Fadli, R., Samala, A. D., Fricticarani, A., Dayurni, P., Rahmadani, K., Astiti, A. D., & Sabir, A. (2023). Exploring the impact of using Chat-GPT on student learning outcomes in technology learning: The comprehensive experiment. Advances in Mobile Learning Educational Research, 859–872. https://doi.org/10.25082/AMLER.2023.02.013
Han, T., & Sari, E. (2022). An investigation on the use of automated feedback in Turkish EFL students’ writing classes. Computer Assisted Language Learning. Advance online publication. https://doi.org/10. 1080/09588221.2022.2067179
Harvey-Scholes, C. (2018). Computer-assisted detection of 90% of EFL student errors. Computer Assisted Language Learning, 31(1), 1-13. https://doi.org/10.1080/09588221.2017.1392322
Hasanah, Y. A., & Fatimah, S. (2020). Undergraduates’ ability in writing paraphrase and their perceptions of plagiarism. Conference: 7th International Conference on English Language and Teaching (ICOELT 2019). https://doi.org/10.2991/assehr.k.200306.019
Hayati, A.M. (2005). The Computer and Language Teaching. The Journal of Asian EFL, 4, 1-8.
Huang, F., Kwak, H., & An, J. (2023). Is ChatGPT better than human annotators? Potential and limitations of ChatGPT in explaining implicit hate speech. arXiv [Preprint]. https://doi.org/10.1145/3543873.3587368
Jacobsen, D.  M., & Lock, J.V. (2005). Technology and teacher education for a knowledge era: Mentoring for student futures, not our past.  Journal of Technology and Teacher Education, 12 (1), pp. 75-87.
Jagdishbhai, N., & Thakkar, K. Y. (2023). Exploring the capabilities and limitations of GPT and Chat GPT in natural language processing. Journal of Management Research and Analysis, 10, 18–20. https://doi.org/10.18231/j.jmra.2023.004
Jahanban-Isfahlan, H., Hadidi Tamjid, N., & Seifoori, Z. (2023). Educational technology in Iranian high schools: EFL teachers’ attitudes, perceived competence, and actual use. Education Research International, https://doi.org/10.1155/2017/9738264
Jiang, L., & Yu, S. (2022). Appropriating automated feedback in L2 writing: Experiences of Chinese EFL student writers. Computer Assisted Language Learning, 35(7), 1329–1353. https://doi.org/10.1080/09588221.2020.1799824
Johnson, K. Pasquale, F. & Chapman, J. (2019). Artificial intelligence, machine learning, and bias in finance: Toward responsible innovation. Fordham Press.  Available at: https://ir.lawnet.fordham.edu/flr/vol88/iss2/5
Katar, O., Özkan, D., GPT, Yildirim, Ö., & Acharya, U. R. (2023). Evaluation of GPT-3 AI language model in research paper writing. Turkish Journal of Science and Technology, 18, 311–318. https://doi.org/10.55525/tjst.1272369
Khazaie, S., Hayati, A. M., & Jalilifar, A. R. (2013). Contribution to mobile-enhanced English language pedagogy among Iranian L2 learners. Journal of English Language Teaching and Learning, 12, 63-93.
Khoso, F. J., Ali, N., & Aslam, N. (2023). Use of Chat-GPT and AI tools by undergraduates: Students and teachers’ perspective. Spry Contemporary Educational Practices, 2(2), 215–238.
Li, Y. (2024). Intelligent english teaching based on the pedagogy of performing another culture and ChatGPT technology. Frontiers in Education Technology, 7, 13. https://doi.org/10.22158/fet.v7n1p13
Lutai, N., & Besarab, T. (2020). Teaching paraphrasing in second language classrooms. Naukovì Zapiski Nacìonalʹnogo Unìversitetu «Ostrozʹka Akademìâ». Serìâ «Fìlologìâ», 9(77), 224–227. https://doi.org/10.25264/2519-2558-2020-9(77)-224-227
Mahapatra, S. (2024). Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study. Smart Learn. Environ. 11, 9. https://doi.org/10.1186/s40561-024-00295-9
Matsuda, P. K. (1998). Situating ESL writing in a cross-disciplinary context. Written Communication, 15, 99-122. https://doi.org/10.1177/0741088398015001004
Meskill, C. (1999). 20 minutes into the future. In J. Egbert & E. Hanson-Smith (Eds.), CALL environments: Research, Practice, and Critical Issues (pp. 459-469). TESOL, Inc
Moghadam, M.H. (2023). Artificial Intelligence and the future of university education in Iran. Quarterly Journal of Research and Planning in Higher Education, 29(1). https://doi.org/10.61838/irphe.29.1.1
Muller, E. & Gregoric, A. (2017). Impact of ChatGPT on learning motivation: Teachers and students’ voices. Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51
Nguyen, T.T.L., & Suwannabubpha, S. (2021). EFL writing at Thai secondary schools: Teachers and students’ views, difficulties and expectations. Language Related Research, 12(3), 187-214. https://doi.org/10.29252/LRR.12.3.7
Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning and Teaching, 6(1), 1 – 22. https://doi.org/10.37074/jalt.2023.6.1.9
Patty, J. (2014). The use of AI in language learning: what you need to know. Jurnal Review Pendidikan Dan Pengajaran, 7(1), 642–654. https://doi.org/10.31004/jrpp.v7i1.24609
Polakova, P., & Ivenz, P. (2024). The impact of ChatGPT feedback on the development of EFL students’ writing skills. Cogent Education, 11(1). https://doi.org/10.1080/2331186X.2024.2410101
Rahimi, M., & Yadollahi, S. (2011). Foreign language learning attitude as a predictor of attitudes towards computer-assisted language learning. Procedia Computer Science, 3, 167-174. http://dx.doi.org/10.1016/j.procs.2010.12.029
Ram, B. & Verma, P. (2023). Artificial intelligence AI-based Chatbot study of ChatGPT, Google AI Bard and Baidu AI. World Journal of Advanced Engineering Technology and Sciences, 8(01), 258–261. https://doi.org/10.30574/wjaets.2023.8.1.0045
Rovikasari, M., S., & Supriyadi, S. (2020). Teachers’ attitudes toward discovery learning: A case study in writing class. International Journal of Educational Research Review, 5(2), 135-140. https://doi.org/10.24331/ijere.686679
Rudolph, J. Tan, S. Tan, S. (2022). ChatGPT: bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning Teaching, 6 (1), 1–22. https://doi.org/10.37074/jalt.2023.6.1.9
Shidiq, M. (2023). The use of artificial intelligence-based chat-GPT and its challenges for the world of education; from the viewpoint of the development of creative writing skills. Proceeding International Conference on Education, Sococity, and Humanity, 1 (1), 353-357.
Slamet, J. (2024). Potential of ChatGPT as a digital language learning assistant: EFL teachers’ and students’ perceptions. Discov Artif Intell 4, 46. https://doi.org/10.1007/s44163-024-00143-2
Su, Y., Lin, Y., & Lai, C. (2023). Collaborating with ChatGPT in argumentative writing classrooms. Assessing Writing, 57, 100752. https://doi.org/10.1016/j.asw.2023.100752
Tafazoli, D. (2024). Exploring the potential of generative AI in democratizing English language education. Computers and Education: Artificial Intelligence, 7. https://doi.org/10.1016/j.caeai.2024.100275
van Dis, E. A., Bollen, J., Zuidema, W., van Rooij, R., & Bockting, C. L. (2023). ChatGPT: five priorities for research. Nature, 614(7947), 224 – 226. https://doi.org/10.1038/d41586-023-00288-7
Wambsganss, T., Janson, A., & Leimeister, J. M. (2022). Enhancing argumentative writing with automated feedback and social comparison nudging. Computers & Education, 191, 104644. https://doi.org/10.1016/j.compedu.2022.104644
Wang, Y. H., & Liao, H. C. (2008). The application of learning portfolio assessment for students in the technological and vocational education system. Asian EFL Journal, 10(2), 132-154.
Wang, B., Hu, S. J., Sun, L., & Freiheit, T. (2020). Intelligent welding system technologies: State-of-the-art review and perspectives. Journal of Manufacturing Systems, 56, 373–391. https://doi.org/10.1016/j.jmsy.2020.06.020
Wilkens, O. (2020).  Artificial intelligence in the workplace – A double-edged sword. International Journal of Information and Learning Technology, https://doi.org/10.1108/IJILT-02-2020-0022
Xue, Y & Wang, Y. (2022). Artificial intelligence for education and teaching. Wireless Communications and Mobile Computing, 10(6),1-10 https://doi.org/10.1155/2022/4750018
Yanning, D. (2017). Teaching and assessing critical thinking in second language writing: An infusion approach. Chinese Journal of Applied Linguistics, 40(4), 431 – 451. https://doi.org/10.1515/cjal-2017-0025
Yusriadi, Y., Rusnaedi, Siregar, N. A., Megawati, S., & Sakkir, G. (2023). Implementation of artificial intelligence in Indonesia. International Journal of Data and Network Science, 7(1), 283– 294. https://doi.org/10.5267/j.ijdns.2022.10.005 
Zaki, N., Turaev, S., Shuaib, K., Krishnan, A., & Mohamed, E. (2023). Automating the mapping of course learning outcomes to program learning outcomes using natural language processing for accurate educational program evaluation. Education and Information Technologies, 28(12), 16723–16742. https://doi.org/10.1007/s10639-023-11877-4
Zolfaghari ‎, F. (2023). Answer changing in online and traditional pen-paper tests: The case ‎of upper intermediate EFL learners with different cognitive styles. Applied Research on English Language, 12(3), 1-22. https://doi.org/10.22108/are.2023.138092.2112
Applied Linguistics Inquiry
Volume 2, Issue 1
March 2024
Pages 89-103

Files

History

  • Receive Date: 07 November 2023
  • Revise Date: 10 December 2024
  • Accept Date: 23 January 2024

Share

How to cite

Statistics