Degree-related writing before and after the public release of ChatGPT: A pilot corpus study of BA and MA theses by L2 English majors

Presentation at ICFSLA2026, International Conference on Foreign and Second Language Acquisition, 28-30 May 2026, Szczyrk, Poland, https://icfsla.us.edu.pl/ 

Abstract

The public release of ChatGPT in November 2022 has created new opportunities and challenges for research and higher education. In university contexts, the impact of generative AI has been discussed in relation to multilingual and multicultural teaching, its augmentative potential in language education, ethical concerns surrounding AI use in writing and assessment, and linguistic differences between AI-generated and human academic writing (Cotton et al., 2024; Nowacki & Wrochna, 2025; Pérez-Paredes et al., 2025; Yu et al., 2025). Against this background, we report on a small-scale study that pilots a project exploring how degree-related writing and research practices have changed following the public availability of LLMs.

The study draws on a corpus of 240 English-language BA and MA theses by L2 English majors in linguistics or literary studies and submitted to the Faculty of Humanities at a large public university in Poland. The material is organised into datasets representing two timeframes: 2021-2022 and 2024-2025, each comprising 60 high- and 60 low-graded theses. The analysis addresses the following questions: (i) What linguistic differences can be observed between theses completed before and after the public release of ChatGPT? (ii) Are there any patterns distinguishing high-graded and passing grade theses across the two periods? Focusing on lexical choice, syntactic complexity, and indicators of textual coherence and logical relations, the analysis employs AntConc (Anthony, 2024), LancsBox (Brezina & Platt, 2024), and Sketch Engine (Kilgarriff et al., 2014). The findings will be interpreted in light of previous research on the linguistic features of AI-generated texts (Herbold et al., 2023; Yildiz Durak et al., 2025) and used to refine the analytical procedures and design qualitative instruments for the full-scale project.

References

Anthony, L. (2024). AntConc 4.3 [Computer Software]. Waseda University, Tokyo, Japan. https://www.laurenceanthony.net/software/AntConc

Brezina, V., & Platt, W. (2024). #LancsBox X [software]. Lancaster University. http://lancsbox.lancs.ac.uk  

Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 61(2), 228–239. https://doi.org/10.1080/14703297.2023.2190148

Herbold, S., Hautli-Janisz, A., Heuer, U., Kikteva, Z., & Trautsch, A. (2023). A large-scale comparison of human-written versus ChatGPT-generated essays. Scientific Reports, 13, 18617. https://doi.org/10.1038/s41598-023-45644-9

Hosseini, M., Gao, P., & Vivas-Valencia, C. (2025). A social-environmental impact perspective of generative artificial intelligence. Environmental Science and Ecotechnology 23, 100520,  https://doi.org/10.1016/j.ese.2024.100520  

Kilgarriff, A., Baisa, V., Bušta, J., Jakubíček, M., Kovář, V., Michelfeit, J., Rychlý, P., & Suchomel, V. (2014). The Sketch Engine: Ten years on. Lexicography, 1, 7-36. https://doi.org/10.1007/s40607-014-0009-9

Nowacki, L., & Wrochna, A. E. (2025). ChatGPT theses. Identifying distinctive markers in AI-generated versus human-created texts: A multimodal analysis in university education. E-Learning and Digital Media, 0(0). https://doi.org/10.1177/20427530251331083

Pérez-Paredes, P., Curry, N., & Ordoñana-Guillamón, C. (2025). Critical AI literacy for applied linguistics and language education students. Journal of China Computer-Assisted Language Learning, 5(2), 175-214. https://doi.org/10.1515/jccall-2025-0005

Yildiz Durak, H., Eğin, F., & Onan, A. (2025), A Comparison of Human-Written Versus AI-Generated Text in Discussions at Educational Settings: Investigating Features for ChatGPT, Gemini and BingAI. European Journal of Education, 60, e70014. https://doi.org/10.1111/ejed.70014

Yu, H., Guo, Y., Yang, H., Zhang, W., & Dong, Y. (2025), Can ChatGPT Revolutionize Language Learning? Unveiling the Power of AI in Multilingual Education Through User Insights and Pedagogical Impact. European Journal of Education, 60: e12749. https://doi.org/10.1111/ejed.12749