Published August 1, 2020 | Version v1
Dataset Open

SemEval-2020 Task 7: Assessing Humor in Edited News Headlines

  • 1. University of Rochester
  • 2. Microsoft Research AI

Contributors

Contact person:

  • 1. University of Rochester

Description

This is the task dataset for SemEval-2020 Task 7: Assessing Humor in Edited News Headlines.

The task’s dataset contains news headlines in which short edits were applied to make them funny, and the funniness of these edited headlines was rated using crowdsourcing. This task includes two subtasks, the first of which is to estimate the funniness of headlines on a humor scale in the interval 0-3. The second subtask is to predict, for a pair of edited versions of the same original headline, which is the funnier version.

CodaLab page hosting the competition:
https://competitions.codalab.org/competitions/20970

Starter Github code (scripts for running baseline and evaluation):
https://github.com/n-hossain/semeval-2020-task-7-humicroedit

Task mailing list:
https://groups.google.com/forum/#!forum/semeval-2020-task-7-all
----------------------------------------------------------------------

ZIP contents:
-------------

Folders:
    - subtask-1: Dataset for the funniness regression subtask.
    - subtask-2: Dataset for the "Funnier of the Two" classification subtask.

Files:
    - {train, dev, test}.csv: the task's dataset including labels
    - train_funlines.csv: additional training data gathered from the FunLines competition (https://funlines.co)
    - baseline.zip: contains csv file which is the output of the BASELINE system. This is a template of the output format that can be submitted to CodaLab for scoring.

Reference

Please cite the task paper when using this dataset:

Nabil Hossain, John Krumm, Michael Gamon and Henry Kautz. 2020. Semeval-2020 Task 7: Assessing Humor in Edited News Headlines. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2020).

BIBTEX: 
@InProceedings{hossainSemEval2020Task7, author = {Hossain, Nabil and Krumm, John and Gamon, Michael and Kautz,Henry}, title = {SemEval-2020 {T}ask 7: {A}ssessing Humor in Edited News Headlines}, booktitle = {Proceedings of the 14th International Workshop on Semantic Evaluation ({S}em{E}val-2020)}, address = {Barcelona, Spain}, year = {2020}}

 

Files

semeval-2020-task-7-dataset.zip

Files (1.6 MB)

Name Size Download all
md5:9953e94c4dc7e50d68b010d68a143f9c
1.6 MB Preview Download

Additional details

References

  • Nabil Hossain, John Krumm, Michael Gamon and Henry Kautz. 2020. Semeval-2020 Task 7: Assessing Humor in Edited News Headlines. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2020).