Published July 18, 2022
| Version v1
Conference paper
Open
Preliminary Experiments with Transformer based Approaches To Automatically Inferring Domain Models from Textbooks
Authors/Creators
Contributors
Editor (2):
- 1. University of Canterbury, NZ
- 2. University of Illinois Urbana–Champaign, US
Description
Domain modeling is a central component in education technologies as it represents the target domain students are supposed to train on and eventually master. Automatically generating domain models can lead to substantial cost and scalability benefits. Automatically extracting key concepts or knowledge components from, for instance, textbooks can enable the development of automatic or semi-automatic processes for creating domain models. We explore in this work the use of transformer based pre-trained models for the task of keyphrase extraction. Specifically, we investigate and evaluate four different variants of BERT, a pre-trained transformer based architecture, that vary in terms of training data, training objective, or training strategy to extract knowledge components from textbooks for the domain of intro-to-programming. We report results obtained using the following BERT-based models: BERT, CodeBERT, SciBERT and RoBERTa.