April 9, 2020 Dataset Open Access

MESINESP: Medical Semantic Indexing in Spanish - Development dataset

Martin Krallinger; Aitor Gonzalez-Agirre; Alejandro Asensio

Introduction

The Mesinesp (Spanish BioASQ track, see https://temu.bsc.es/mesinesp) development set has a total of 750 records indexed manually by seven experienced medical literature indexers. Indexing is done using DeCS codes, a sort of Spanish equivalent to MeSH terms. Records were distributed in a way that each article was annotated, at least, by two different human indexers.

The data annotation process consisted in two steps:

1. Manual indexing step. DeCS codes were manually assigned to each record following the DeCS manual indexing guidelines.
2. Manual validation and consensus. The joined set of manually indexed DeCS codes generated by both indexers were manually revised and corrections were done.

These annotations were analyzed, resulting in an agreement using the Jaccard index.

Records consisted basically in medical literature abstracts and titles from the IBECS and LILACS databases.

Zip structure
The zip file contains two different development sets:

• Official development set, which has the union of the annotations, with an agreement of macro = 0.6568 and micro = 0.6819. This set is composed by all the different (unique) DeCS codes that have been added by any annotator for each document; and
• Core-descriptors development set, which has the intersection of the annotations, with an agreement of macro = 1.0 and micro = 1.0. This set is composed of the common DeCS codes that have been added by two or more annotators for each document.

Corpus format

Each dataset is a JSON object with one single key named "articles", which contains a list of documents. So, the raw format of the file is one line per document plus two additional lines (the first and the last) to enclose that list of documents and the expected type of data is as follows:

{"articles":[
{"abstractText":str,"db":str,"decsCodes":list,"id":str,"journal":str,"title":str,"year":int},
...
]}

To clarify, the order of appearance of the fields in each document is as follows (note that this example it is pretty printed for readability purposes):

{
  "articles": [
    {
      "abstractText": "Content of the abstract",
      "db": "Name of the source database",
      "decsCodes": [
        "code1",
        "code2",
        "code3"
      ],
      "id": "Id of the document",
      "journal": "Name of the journal",
      "title": "Title of the document",
      "year": 2019
    }
  ]
}

Note: The fields "db", "journal" and "year" might be null.