Dataset for Evaluating Abstractive Summaries of Crisis-Related Social Media

The dataset created for evaluation of summaries generated from social media posted during five natural disasters.

The dataset contains:

• ground truth reports created by human assessor based on ERCC Echo Flash reports (5 events);

• summaries generated by extractive and abstractive state-of-the-art with manual annotation of category-relevant and crisis-relevant claim.

The dataset with annotated social media postings — link

The full description of used summarization methods, sources of data and evaluation metrics available in the paper — link