Conference paper Open Access

Text Simplification from Professionally Produced Corpora

Carolina Scarton; Gustavo Henrique Paetzold; Lucia Specia

The lack of large and reliable datasets has been hindering progress in Text Simplification (TS). We investigate the application of the recently created Newsela corpus, the largest collection of professionally written simplifications available, in TS tasks. Using new alignment algorithms, we extract 550,644 complex-simple sentence pairs from the corpus. This data is explored in different ways: (i) we show that traditional readability metrics capture surprisingly well the different complexity levels in this corpus, (ii) we build machine learning models to classify sentences into complex vs. simple and to predict complexity levels that outperform their respective baselines, (iii) we introduce a lexical simplifier that uses the corpus to generate candidate simplifications and outperforms the state of the art approaches, and (iv) we show that the corpus can be used to learn sentence simplification patterns in more effective ways than corpora used in previous work.

Files (242.5 kB)
Name Size
242.5 kB Download
All versions This version
Views 100100
Downloads 3636
Data volume 8.7 MB8.7 MB
Unique views 8080
Unique downloads 3333


Cite as