Dataset Open Access

Same Side Stance Classification Adversarial Test Cases

Ahmad Dawar Hakimi; Erik Körner; Gregor Wiedemann; Gerhard Heyer; Martin Potthast

This dataset contains 175 cases. It is manually created to reveal the ability of models to solve different types of adversarial cases for same side stance predictions more systematically. The examples selected here are derived from the dataset used in the Same Side Stance Classification shared task.

We have selected 25 distinct arguments from the "gay marriage" topic that are short and express their stance clearly. For each selected argument, we construct new arguments of four distinct types to obtain two pairs, one with the same stance, and one with an opposing stance:

  • Negation: a simple negation of the argument.
  • Paraphrase: alters important words from the argument to synonymous expressions with the same stance.
  • Argument: uses an argument from the same topic and stance, but semantically completely different regarding the first one.
  • Citation: repeats or summarizes the first argument and then expresses agreement or rejection (a case frequently occurring in the dataset).

The types Paraphrase, Argument, and Citation are also formulated in a negated version to create additional test instances for the opposite stance.

Files (48.9 kB)
Name Size
artificial_evalset.tsv
md5:904e1e2857dc6100d0e309851c52c41c
48.9 kB Download
  • Benno Stein, Yamen Ajjour, Roxanne El Baff, Khalid Al-Khatib, Philipp Cimiano, and Henning Wachsmuth. Same Side Stance Classification. In Yamen Ajjour et al, editors, Same Side Stance Classification Shared Task 2019, volume 2921, July 2021

60
3
views
downloads
All versions This version
Views 6060
Downloads 33
Data volume 146.8 kB146.8 kB
Unique views 4444
Unique downloads 33

Share

Cite as