Published July 18, 2022 | Version v1

Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks

Contributors

  • 1. University of Canterbury, NZ
  • 2. University of Illinois Urbana–Champaign, US

Description

Domain modeling is a central component in education technologies as it represents the target domain students are supposed to train on and eventually master. Automatically generating domain models can lead to substantial cost and scalability benefits. Automatically extracting key concepts or knowledge components from, for instance, textbooks can enable the development of automatic or semi-automatic processes for creating domain models. We explore in this work the use of transformer based pre-trained models for the task of keyphrase extraction. Specifically, we investigate and evaluate four different variants of BERT, a pre-trained transformer based architecture, that vary in terms of training data, training objective, or training strategy to extract knowledge components from textbooks for the domain of intro-to-programming. We report results obtained using the following BERT-based models: BERT, CodeBERT, SciBERT and RoBERTa.

Files

2022.EDM-posters.79.pdf

Files (282.5 kB)

Name Size
md5:5fcf3da842eab522758c26fc713a56e4
282.5 kB Preview Download

Additional details