Assessing Student Writing Assignments with Large Language Models

This is a poster presented at the 6th Shaw-IAU Workshop on Astronomy for Education, organised by the IAU Office of Astronomy for Education (OAE, http//astro4edu.org).

Title: 
Assessing Student Writing Assignments with Large Language Models

Presenter:
Wenger, Matthew (The University of Arizona)

Writing assignments are useful for promoting and assessing student learning. It is difficult to implement student writing in large classes, and nearly impossible to provide iterative feedback. For large online classes, such as Massive Open Online Classes (MOOCs), the only solution has been to use peer graders. Unfortunately peer grading can be unreliable and peer graders do not always leave useful feedback. We used Large Language Models (LLMs) to test whether LLM's can accurately grade student writing assignments and provide feedback. Our results show that LLMs can provide accurate feedback similar to instructors. We also found that the grades assigned by LLMs were consistent with instructor scores, and more accurate than peer graders.

Collaborators:

About the 6th Shaw-IAU Workshop:

The 6th Shaw-IAU Workshop took place:12 - 15 November 2024. The Shaw-IAU workshops focus on astronomy education for primary and secondary school students and teacher training both in universities and in service. This year's workshop featured two special topics: an exploration of the first two years of the James Webb Space Telescope for the scientific topic, and the crucial aspect of evaluation in educational contexts as the non-scientific topic.

More details can be found on: https://astro4edu.org/shaw-iau/

Keep up to date with future Shaw-IAU Workshops and other opportunities at the IAU Office of Astronomy for Education by joining our mailing list https://astro4edu.org/mailing-list/

Follow the IAU OAE on Bluesky and Facebook under @astro4edu