Assisted Text Annotation Using Active Learning toAchieve High Quality with Little Effort

arge amounts of annotated data have become moreimportant than ever, especially since the rise of deep learningtechniques. However, manual annotations are costly. We proposea tool that enables researchers to create large, high-quality,annotated datasets with only a few manual annotations, thusstrongly reducing annotation cost and effort. For this purpose,we combine an active learning (AL) approach with a pre-trained language model to semi-automatically identify annotationcategories in the given text documents. To highlight our researchdirection’s potential, we evaluate the approach on the task ofidentifying frames in news articles. Our preliminary results showthat employing AL strongly reduces the number of annotationsfor correct classification of even these complex and subtle frames.On the framing dataset, the AL approach needs only 16.3% of theannotations to reach the same performance as a model trainedon the full dataset.