Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.

There is a newer version of the record available.

Published March 14, 2021 | Version v1
Dataset Restricted

Profiling Hate Speech Spreaders on Twitter

  • 1. SYMANTO RESEARCH
  • 2. UNIVERSITAT POLITÈCNICA DE VALÈNCIA
  • 3. UNIVERSITÀ DEGLI ESTUDI DI MILANO - BICOCCA

Description

Task

Hate speech (HS) is commonly defined as any communication that disparages a person or a group on the basis of some characteristic such as race, colour, ethnicity, gender, sexual orientation, nationality, religion, or other characteristics. Given the huge amount of user-generated contents on Twitter, the problem of detecting, and therefore possibly contrasting the HS diffusion, is becoming fundamental, for instance for fighting against misogyny and xenophobia. To this end, in this task, we aim at identifying possible hate speech spreaders on Twitter as a first step towards preventing hate speech from being propagated among online users.

After having addressed several aspects of author profiling in social media from 2013 to 2020 (fake news spreaders, bot detection, age and gender, also together with personality, gender and language variety, and gender from a multimodality perspective), this year we aim at investigating if it is possible to discriminate authors that have shared some hate speech in the past from those that, to the best of our knowledge, have never done it.

As in previous years, we propose the task from a multilingual perspective:

  • English
  • Spanish

NOTE: Although we recommend participating in both languages (English and Spanish), it is possible to address the problem just for one language.

Award

We are happy to announce that the best performing team at the 9th International Competition on Author Profiling will be awarded 300,- Euro sponsored by Symanto

Data

Input

The uncompressed dataset consists of a folder per language (en, es). Each folder contains:

  • An XML file per author (Twitter user) with 100 tweets. The name of the XML file corresponding to the unique author id.
  • A truth.txt file with the list of authors and the ground truth.

The format of the XML files is:

    <author lang="en">
        <documents>
            <document>Tweet 1 textual contents</document>
            <document>Tweet 2 textual contents</document>
            ...
        </documents>
    </author>
      

The format of the truth.txt file is as follows. The first column corresponds to the author id. The second column contains the truth label.

    b2d5748083d6fdffec6c2d68d4d4442d:::0
    2bed15d46872169dc7deaf8d2b43a56:::0
    8234ac5cca1aed3f9029277b2cb851b:::1
    5ccd228e21485568016b4ee82deb0d28:::0
    60d068f9cafb656431e62a6542de2dc0:::1
    ...
    

Output

Your software must take as input the absolute path to an unpacked dataset, and has to output for each document of the dataset a corresponding XML file that looks like this:

    <author id="author-id"
        lang="en|es"
        type="0|1"
    />
                              

The naming of the output files is up to you. However, we recommend using the author-id as filename and "XML" as an extension.

IMPORTANT! Languages should not be mixed. A folder should be created for each language and place inside only the files with the prediction for this language.

Evaluation

The performance of your system will be ranked by accuracy. For each language, we will calculate individual accuracies in discriminating between the two classes. Finally, we will average the accuracy values per language to obtain the final ranking.

Related Work

Files

Restricted

The record is publicly accessible, but files are restricted to users with access.

Request access

If you would like to request access to these files, please fill out the form below.

You need to satisfy these conditions in order for this request to be accepted:

You may request access to the files in this upload, provided that you fulfil the conditions below. The decision of whether to grant/deny access is solely under the responsibility of the record owner.

You are currently not logged in. Do you have an account? Log in here