MultiClinAI corpus: Multilingual Clinical Entity Annotation Projection and Extraction
Authors/Creators
- 1. Barcelona Supercomputing Center
Description
Please cite if you use this dataset:
Gallego-Donoso, F., Lima-López, S., Rosell, J., Farré-Maduell, E., Krallinger, M. (2026). The MultiClinAI Shared Task on Multilingual Clinical Corpus Construction and Concept Extraction: Systems, Evaluation, and Datasets. Proceedings of the 11th Social Media Mining for Health (#SMM4H) and Health Real-World Data (HeaRD) Workshop and Shared Tasks.
@inproceedings{multiclinai-overview-2026,
title={The MultiClinAI Shared Task on Multilingual Clinical Corpus Construction and Concept Extraction: Systems, Evaluation, and Datasets},
author={Donoso, Fernando Gallego and Lima-L{\'o}pez, Salvador and Rosell, Judith and Farr{\'e}-Maduel, Eul{\`a}lia and Krallinger, Martin},
booktitle={Proceedings of the 11th Social Media Mining for Health Research and Applications (SMM4H-HeaRD 2026) Workshop and Shared Tasks},
pages={309--331},
year={2026}
}
v1.3 2026/03/30: The data for the MultiClinCorpus subtask's test and background set are now included in the files; for this task, in addition to the texts in each language, you are provided with a set of documents in Spanish and their annotations. Using the Spanish annotations as a basis, you have to find the equivalent annotations in each language (annotation projection).
The MultiClinAI (Multilingual Clinical Entity Annotation Projection and Extraction) shared task challenges participants to create systems that can automatically create multilingual versions of Gold Standard corpora from a seed language (in our case, Spanish) to six different target languages (Czech, Dutch, English, Italian, Romanian and Swedish). This is a process known as annotation projection. In parallel, participants are also challenged to create systems for clinical concept extraction in the seven languages of the task. This needs to be done for three different clinical information types: DISEASE, SYMPTOM and PROCEDURE.
This repository currently includes the Training Data of the task, which includes part of the DisTEMIST, SympTEMIST and MedProcNER corpora, as well as the CardioCCC corpus (expanded from its original use in the MultiCardioNER shared task), in the seven languages mentioned above. The Test Set (only texts) and Background Set for the both subtasks are also included. For more information about the creation process and context of this dataset, please visit the Data section of the task's website (linked below). As a note, each dataset in each language contains a slightly different version of the text due to translation revisions made during the annotation projection process, meaning they cannot be used for multilabel approaches.
In short, the task is divided into two subtracks:
- Sub-task MultiClinNER (Multilingual Comparable Clinical Entity Recognition). This is a common Named Entity Recognition task; using the texts in each language, try to extract the entities contained in each text (Spanish, Czech, Dutch, English, Italian, Romanian and Swedish).
- Sub-task MultiClinCorpus (Multilingual Comparable Clinical Corpus Generation). This is an annotation projection task; starting from the original Spanish, try to obtain the annotations in the other languages (Czech, Dutch, English, Italian, Romanian and Swedish).
Both tasks will be evaluated using common classification metrics: precision, recall and F-1. An official evaluation library will be released soon. Also in both sub-tasks, teams can submit results for any target language. Submitting for all languages is not mandatory. Participants are free to create their systems in any way they want (i.e. monolingual or multilingual models, word alignment or generative models, unilabel or multilabel, …), and the use of creative solutions is encouraged.
File structure:
MultiClinNER/
├── MultiClinNER-es/
│ ├── MultiClinNER-es-train/
│ │ ├── MultiClinNER-es-train-disease/
│ │ │ ├── MultiClinNER-es-train-disease-00001.txt
│ │ │ ├── MultiClinNER-es-train-disease-00002.txt
│ │ │ ├── MultiClinNER-es-train-disease-00003.txt
│ │ │ ├── ...
│ │ ├── MultiClinNER-es-train-symptom/
│ │ │ ├── ...
│ │ ├── MultiClinNER-es-train-procedure/
│ │ │ ├── ...
├── MultiClinNER-cz/
│ ├── MultiClinNER-cz-train/
│ │ ├── MultiClinNER-cz-train-disease/
│ │ │ ├── MultiClinNER-cz-train-disease-00001.txt
│ │ │ ├── ...
│ │ ├── MultiClinNER-cz-train-symptom/
│ │ │ ├── ...
│ │ ├── MultiClinNER-cz-train-procedure/
│ │ │ ├── ...
├── MultiClinNER-{nl,en,it,ro,sv}/ (same structure as es and cz)
... (same for MultiClinCorpus folder)
Resources
Additional resources and corpora
At the NLP for Biomedical Information Analysis group (formerly Text Mining Unit), one of our missions is the open publication of datasets to train and benchmark biomedical information extraction, normalization and indexing systems. For that reason, we have released multiple datasets as part of shared tasks over the years. If you are interested in MultiClinAI, you might want to take a look at some of our resources and competitions about:
- Clinical content extraction: DisTEMIST (diseases), MedProcNER/ProcTEMIST (clinical procedures), SympTEMIST (signs and findings), CANTEMIST (tumour morphology), CodiEsp (coding to ICD), PharmaCoNER (chemicals and proteins), LivingNER (species and humans), MultiCardioNER (diseases and medications, includes the DrugTEMIST corpus as well as cardiology-specific data)
- Socio-demographic / Social Determinants of Health content extraction: MEDDOPLACE (locations and more) MEDDOCAN (sensitive data), MEDDOPROF (occupations), ToxHabits (extraction of substance use-related content)
- Information extraction in social media: SocialDisNER (diseases), ProfNER (occupations)
- Linguistic aspects: BARR1 and BARR2 (abbreviation resolution)
- Machine Translation: ClinSpEn (EN<->ES clinical content translation)
- Summarization: MultiClinSUM (multilingual summarization of clinical content)
Contact:
- Salvador Lima-López (<salvador [dot] limalopez [at] gmail [dot] com>)
- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)
License: This work is licensed under a Creative Commons Attribution 4.0 International License.