TAXODIS - A Taxonomy of Online Disinformation

The Taxonomy of Online Disinformation (TAXODIS) structures a set of well-defined and analysed linguistic features of online disinformation discourse and is meant to help annotate training data to nourish machine learning and computational models.

TAXODIS has been implemented as a SKOS vocabulary/thesaurus and is available as an RDFS resource (in TTL/Turtle format) at: 

https://hop.fzi.de/wordpress/wp-content/uploads/taxodis/v1.0/TAXODIS.ttl

The (resolvable) namespace of the taxonomy is https://hop.fzi.de/taxodis/.

PURPOSE

The taxonomy can be used for the fine-grained annotation of disinformation datasets and for the publication of the annotation results as Linked (Open) Data. It can be used by automated detection systems to train classification models.

STRUCTURE

The taxonomy contains 66 concepts organized in a hierarchical structure. Its top concept is "disinformation characteristic", which describes characteristics that are indicative of disinformation in a piece of content. This top term has three narrower terms: 

• "detection feature", which classifies the piece of content based on linguistic or stylistic features that are indicative of the detection of disinformation (e.g. length of the headline, lexical and contentual poorness, level of semantic incoherence, lack of new information, level of topicality, etc.),
• "categorization", which classifies the piece of content based on its theme or content type. e.g. social (theme), conspiracy theory (content type), and 
• "veracity", which classifies the piece of content based on its veracity, e.g. mostly false, mixture, etc. 

The full hierarchy is shown in this figure.

USAGE

The taxonomy can be used together with existing, established vocabularies for the annotation of (disinformation) resources. We suggest the exploitation of the Web Annotation Data Model; a W3C recommendation for the structured representation of annotations that can be shared and reused across different platforms. 

This figure shows an example in which an article (instance of class oa:Target) is linked to two annotations: one which categorises the article as of social theme (taxodis:themsoc) and one which categorises the article as having "high" topicality level (taxodis:mobtopical). The intension (motivation) of both annotations is classification (oa:classifying). Notice that the first annotation is directly linked to the taxonomy term taxodis:themsoc through multiple instantiation (the term is an instance of both oa:Body and skos:Concept). This annotation method can be applied for all taxonomy terms that are under "categorisation" and "veracity", since these terms do not accept a degree value or level like the terms that are under "detection feature".

This figure shows how we can link the annotated resource with rich (meta)data using another established vocabulary, namely schema.org. The annotated article is both an instance of oa:Target and an instance of schema:CreativeWork. This allows using properties of schema.org for providing more information about the article, such as its URL (instance of schema:URL), its publication date (instance of schema:DateTime), its headline (instance of schema:Text), its author (instance of schema:Person), and its content (instance of schema:Text). We can also link the article with entities of different types mentioned in it, such as persons, places, etc., using the property schema:mentions. 
In addition, we can link claims (instances of schema:Claim) to the articles using the property schema:appearance. A claim can be then linked to its text, video/audio (if any) and author (using the properties schema:text, schema:video/schema:audio, and schema:author, respectively), as well as with claim reviews (instances of schema:ClaimReview). In a similar way, a claim review can be linked with related data such as its author, URL, publication date, headline, review body, etc.

Queries that can be answered using TAXODIS annotations include:

• Retrieve all resources classified as of social theme and which have a high level of emotional polarization
• Retrieve all resources with imposter content together with the values of all features that are under "psychology feature"
• Retrieve the number of resources per content type having high usage of emphatic wording
• Retrieve all resources published on a specific time period containing claims that have been reviewed and have received a veracity score "mostly false"
• Retrieve all resources mentioning a specific person which are mostly false, together with the values of all features that are under "detection feature"

The first query of the above list is translated to SPARQL as follows:

PREFIX taxodis: <https://hop.fzi.de/taxodis/>
PREFIX oa: <http://www.w3.org/ns/oa#>
PREFIX schema: <http://schema.org/>
SELECT ?resourceUri ?resourceHeadline ?resourceAuthor 
WHERE {
   ?annot1 oa:hasTarget ?resourceUri ; oa:hasBody taxodis:themsoc .
   ?annot2 oa:hasTarget ?resourceUri ; oa:hasBody ?annot2Body .
   ?annot2Body oa:type taxodis:mobpolar ; rdf:value "high" .
   OPTIONAL { ?resourceUri schema:headline ?resourceHeadline } 
   OPTIONAL { ?resourceUri schema:author ?resourceAuthor } 
}

ANNOTATION PROTOCOL / GUIDELINES

We do not provide specific annotation guidelines as of how to use the taxonomy, because such protocol is usually highly task and domain dependent. Thus, we leave to the specific community and teams the design of annotation guidelines according to their needs. It is worth noting that some taxonomy categories, such as "emotional polarization" versus "sensationalism", exhibit conceptual proximity that may challenge consistent annotation. In such cases, multiple labels to a single content item can be assigned, even within the same dimension where categories are not strictly mutually exclusive. For example, content may be simultaneously labeled as both "clickbait" and "propaganda" when relevant. This multilabel approach reflects the complex and often overlapping nature of disinformation phenomena.

In the context of the DeFaktS project, a  group of interested researchers (from sociology, computer science, and political science) and practitioners (from NGOs and industry) utilized an earlier version of the taxonomy to create labels for identifying different types of disinformation. The participants were provided with a handout of the taxonomy and applied it to social media posts derived from various platforms (mainly Telegram and X). This handout may also prove useful in similar manual annotation efforts.

RELATED PUBLICATION

(currently under review)