PharmaCoNER corpus: gold standard annotations of Pharmacological Substances, Compounds and proteins in Spanish clinical case reports

Intro:

The PharmaCoNER corpus (divided into train, dev and test) is a Gold Standard manually annotated dataset used for the the PharmaCoNER shared task posed at BIONLP-ST (at EMNLP). In addition, we include here the PharmaCoNER background set. It contains the train, development and test sets of the two subtasks (subtask-1 and subtask-2) with Gold Standard annotations. In addition, it contains the documents of the background set, without annotations.

The PharmaCoNER corpus consists of:

• Manually classified clinical case sections derived from Open access Spanish medical publications, named the Spanish Clinical Case Corpus (SPACCC).
• It was manually selected by a practicing oncologist and revised by a clinical documentalist to assure that records were relevant/representative and resembled structure and content relevant to process clinical records.
• The final corpus: 1000 clinical cases 16,504 sentences
• The corpus contains a total of 396,988 words, with an average of 396.2 words per clinical case.
• It covers a range of medical disciplines including oncology, urology, cardiology, pneumology or infections diseases, etc.
• The corpus has been annotated at the mention level by experts in medicinal chemistry and pharmacology following a granular annotation scheme covering four mention types:
  • Entity type 1 (NORMALIZABLES): mentions of chemicals that can be manually normalized to a unique concept identifier (primarily SNOMED-CT).
  • Entity type 2 (NO_NORMALIZABLES): mentions of chemicals that could not be normalized manually to a unique concept identifier.
  • Entity type 3 (PROTEINAS): mentions of proteins/genes following an adaptation of the BioCreative GPRO track annotation guidelines (includes peptides, peptide hormones & antibodies).
  • Entity type 4 (UNCLEAR ): cases of general substance class mentions of clinical relevance, including certain pharmaceutical formulations, general treatments, chemotherapy programs, and vaccines.
  • Mentions class "UNCLEAR" (not evaluated for the PharmaCoNER track)

Please, cite: 

A. G. Agirre, M. Marimon, A. Intxaurrondo, O. Rabal, M. Villegas, M. Krallinger, Pharmaconer: Pharmacological substances, compounds and proteins named entity recognition track, in: Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, 2019, pp. 1–10.

Annotation quality

Inter-annotator agreement: 93% for annotation, 73% for mapping.

For more information, see the paper.

Format

For subtask 1 annotations are distributed in Brat format. (More info at Brat webpage https://brat.nlplab.org/standoff.html)

For subtask-2, codes are associated with each document are given in a TSV file with the following columns: 

filename    code

Shared task goal:

In the two subtasks, the goal is to predict the annotations of the test files (either the ANN files or the TSV with the codes) given only the plain text files. 

Resources:

• Web
• Citation: A. G. Agirre, M. Marimon, A. Intxaurrondo, O. Rabal, M. Villegas, M. Krallinger, Pharmaconer: Pharmacological substances, compounds and proteins named entity recognition track, in: Proceedings of The 5th Workshop on BioNLP Open Shared Tasks, 2019, pp. 1–10.
• Silver Standard corpus
• Annotation guidelines
• PharmaCoNER tagger
• Youtube video(general setting)
• Slides PharmacoNER overview talk at BIONLP-ST / EMNLP 

For further information, please visit https://temu.bsc.es/pharmaconer/ or email us at encargo-pln-life@bsc.es

Copyright (c) 2018 Secretaría de Estado para el Avance Digital (SEAD)

License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Contact

If you have any questions or suggestions, please contact us at:

- Martin Krallinger (<krallinger [dot] martin [at] gmail [dot] com>)

Additional resources and corpora

If you are interested in PharmaCoNER, you might want to check out these corpora and resources:

• DisTEMIST (Corpus of disease mentions and normalization to SNOMED CT, same document collection)
• MedProcNER (Corpus of clinical procedure mentions and normalization to SNOMED CT, same document collection)
• SympTEMIST (Corpus of symptoms, signs and findings mentions and normalization, same document collection)
• MEDDOPROF (Corpus of mentions of professions, occupations and working status and normalization, different document collection with some overlapping documents)
• MEDDOPLACE (Corpus of mentions of place-related entity mentions, including departments, nationalities or patient movements etc.. and normalization, different document collection with some overlapping documents)
• MEDDOCAN (Corpus of mentions of Personal Health Identifiers (PHI), modified synthetic verions of the document collection)
• CANTEMIST (Corpus of cancer tumor morphology mentions and normalization, different document collection)
• CodiESp (Corpus of clinical case reportes with assigned clinical codes from ICD10, Spanish version, same document collection)
• LivingNER (Corpus of mentions of species, including human/family members, pathogens, food, etc.. and normalization to NCBI Taxonomy, different document collection with some overlapping documents)
• SPACCC-POS (Corpus of clinical case reports in Spanish annotated with POS-tags, same document collection)
• SPACCC-TOKEN (Corpus of clinical case reports in Spanish annotated with token-tags (word mention boundaries), same document collection)
• SPACCC-SPLIT (Corpus of clinical case reports in Spanish annotated with sentence boundary-tags, same document collection)
• MESINESP-2 (Corpus of manually indexed records with DeCS /MeSH terms comprising scientific literature abstracts, clinical trials, and patent abstracts, different document collection)