Automated Grammar Scoring with AI Chatbots

Artificial intelligence (AI) continues to revolutionize the way that we work and learn and its impact on education is rapidly unfolding. Recent research is shedding light on how AI can best support teachers and learners. Studies on generative AI and large language models (LLMs) indicate that educators are optimistic AI will play a productive role in education (Polak et al., 2022; Caines et al., 2023). Research in the language learning field has demonstrated that AI is as good or even better at assessing writing than humans (Bridgeman et al., 2012). To continue the investigation of how AI can facilitate automated essay scoring (AES) and language learning in general, this study examines several of the latest AI Chatbots to better understand how accurately and reliably they rate the grammar of writing samples of EFL engineering students in Japan.