This paper presents an overview of the second edition of HIPE (Identifying Historical People, Places and other Entities), a shared task on named entity recognition and linking in multilingual historical documents. Following the success of the first CLEF-HIPE-2020 evaluation lab, HIPE-2022 confronts systems with the challenges of dealing with more languages, learning domain-specific entities, and adapting to diverse annotation tag sets. This shared task is part of the ongoing efforts of the natural language processing and digital humanities communities to adapt and develop appropriate technologies to efficiently retrieve and explore information from historical texts. On such material, however, named entity processing techniques face the challenges of domain heterogeneity, input noisiness, dynamics of language, and lack of resources. In this context, the main objective of HIPE-2022, run as an evaluation lab of the CLEF 2022 conference, is to gain new insights into the transferability of named entity processing approaches across languages, time periods, document types, and annotation tag sets. Tasks, corpora, and results of participating teams are presented. Compared to the condensed overview, this paper contains more refined statistics on the datasets, a break down of the results per type of entity, and a discussion of the `challenges' proposed in the shared task.
\n\n\n\n
For code and data, see the HIPE-eval github organisation: https://github.com/hipe-eval
", "publication_date": "2022-08-10", "publisher": "Zenodo", "related_identifiers": [ { "identifier": "http://ceur-ws.org/Vol-3180/#paper-83", "relation_type": { "id": "isidenticalto", "title": { "de": "Ist identisch mit", "en": "Is identical to" } }, "resource_type": { "id": "publication-conferencepaper", "title": { "de": "Konferenzbeitrag", "en": "Conference paper" } }, "scheme": "url" }, { "identifier": "10.5281/zenodo.6375600", "relation_type": { "id": "describes", "title": { "de": "Beschreibt", "en": "Describes" } }, "resource_type": { "id": "dataset", "title": { "de": "Datensatz", "en": "Dataset" } }, "scheme": "doi" }, { "identifier": "10.5281/zenodo.6045662", "relation_type": { "id": "documents", "title": { "de": "Dokumentiert", "en": "Documents" } }, "resource_type": { "id": "publication-technicalnote", "title": { "de": "Technical Note", "en": "Technical note" } }, "scheme": "doi" } ], "resource_type": { "id": "publication-conferencepaper", "title": { "de": "Konferenzbeitrag", "en": "Conference paper" } }, "rights": [ { "description": { "en": "The Creative Commons Attribution license allows re-distribution and re-use of a licensed work on the condition that the creator is appropriately credited." }, "icon": "cc-by-icon", "id": "cc-by-4.0", "props": { "scheme": "spdx", "url": "https://creativecommons.org/licenses/by/4.0/legalcode" }, "title": { "en": "Creative Commons Attribution 4.0 International" } } ], "subjects": [ { "subject": "Named entity recognition and classification" }, { "subject": "Entity linking" }, { "subject": "Historical texts" }, { "subject": "Information extraction" }, { "subject": "Digitised newspapers" }, { "subject": "Digital humanities" }, { "subject": "Evaluation" }, { "subject": "Classical commentaries" } ], "title": "Extended Overview of HIPE-2022: Named Entity Recognition and Linking in Multilingual Historical Documents", "version": "1.0" }, "parent": { "access": { "owned_by": { "user": "44474" }, "settings": { "accept_conditions_text": null, "allow_guest_requests": false, "allow_user_requests": false, "secret_link_expiration": 0 } }, "communities": { "entries": [ { "access": { "member_policy": "open", "members_visibility": "public", "record_submission_policy": "open", "review_policy": "open", "visibility": "public" }, "children": { "allow": false }, "created": "2019-01-10T15:49:30.817082+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "62edd9a9-6173-4379-9408-21a4bff4269f", "links": {}, "metadata": { "curation_policy": "all outputs related to the newseye projects, including entries from internal and external (project team members) alike.
\r\n", "funding": [ { "award": { "id": "00k4n6c32::770299" }, "funder": { "id": "00k4n6c32" } } ], "page": "NewsEye, funded by the European Union’s Horizon 2020 research and innovation programme, is a research project advancing the state of the art and introducing new concepts, methods and tools for digital humanities by providing enhanced access to historical newspapers for a wide range of users. With the tools and methods created by Newseye, crucial user groups will be able to investigate views and perspectives on historical events and development and, as a consequence, the project will change the way European digital heritage data is (re)searched, accessed, used and analysed.
", "title": "NewsEye" }, "parent": { "access": { "member_policy": "open", "members_visibility": "restricted", "record_submission_policy": "open", "review_policy": "closed", "visibility": "public" }, "children": { "allow": true }, "created": "2022-11-23T15:53:29.436323+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "f0a8b890-f97a-4eb2-9eac-8b8a712d3a6c", "links": {}, "metadata": { "curation_policy": "The EU Open Research Repository serves as a repository for research outputs (data, software, posters, presentations, publications, etc) which have been funded under an EU research funding programme such as Horizon Europe, Euratom or earlier Framework Programmes.
\nThe community is managed by CERN on behalf of the European Commission.
\nZenodo’s general policies and Terms of Use apply to all content.
\nThe EU Open Research Repository accepts all digital research objects which is a research output stemming from one of EU’s research and innovation funding programmes. The funding programmes currently include:
\nHorizon Europe (including ERC, MSCA), earlier Framework Programmes (eg Horizon 2020) as well as Euratom.
\nIn line with the principle as open as possible, as closed as necessary both public and restricted content is accepted. See note on how Zenodo handles restricted content.
\nEU programme beneficiaries are eligible to submit content to the community. The community supports three types of content submissions:
\nSubmission via an EU Project Community (through user interface or programmatic APIs).
\nSubmission directly to the EU Open Research Repository.
\nAutomated harvesting from existing Zenodo content.
\nA representative of an EU project may request an EU Project Community and invite other project participants as members of the community. The project community is linked to one or more European Commission grants. All records in the project community are automatically integrated into the EU Open Research Repository immediately upon acceptance into the project community.
\nAny user may submit a record directly to the EU Open Research Repository. The submission will be moderated by Zenodo staff for compliance with the minimal required metadata requirements and its correctness.
\nRecords found among Zenodo’s existing content will on a regular basis automatically be integrated if they are found to comply with the requirements. The submissions through this method are integrated into the EU Open Research Repository with delay in a fully automated way.
\nRecords in the EU Open Research Repository are required to comply with the following minimal metadata requirements:
\nVisibility: Both public and restricted (with or without embargo and/or access request)
\nResource types: All resource types.
\nLicenses: Public and embargoed records MUST specify a license. The chosen license SHOULD be compliant with the Horizon Europe open science requirements (see Open Science in Horizon Europe)
\nFunding information: Records MUST specify at least one grant from the European Commission.
\nCreators: Creators SHOULD be identified with a persistent identifier (e.g. ORCID, GND, …), and affiliations SHOULD be identified with a persistent identifier (e.g. ROR, ISNI, …)
\nSubjects: Records SHOULD specify one or more fields of science from the European Science Vocabulary.
\nThese metadata requirements comes from the related open science requirements in Horizon Europe which are detailed in each project's grant agreement.
\nAll submissions undergo automated curation checks for compliance with the policy. Submissions through project communities are in addition reviewed by the project community. Submission directly to the EU Open Research Repository is in addiotn reviewed by Zenodo staff.
\nCommunity curators may at any point edit metadata of the records in the community without notice through human or automated processing. The curators may at their sole discretion remove records from the community that are deemed not to comply with the content and curation policy or which are deemed of insufficient quality.
\nThe content and curation policy is subject to change by the community owner at any time and without notice, other than through updating this page.
", "description": "Open repository for EU-funded research outputs from Horizon Europe, Euratom, and earlier Framework Programmes.", "funding": [ { "funder": { "id": "00k4n6c32" } } ], "organizations": [ { "id": "00k4n6c32" } ], "page": "The EU Open Research Repository is a Zenodo-community dedicated to fostering open science and enhancing the visibility and accessibility of research outputs funded by the European Union. The community is managed by CERN on behalf of the European Commission.
\nThe mission of the repository is to support the implementation of the EU's open science policy, providing a trusted and comprehensive space for researchers to share their research outputs such as data, software, reports, presentations, posters and more. The EU Open Research Repository simplifies the process of complying with open science requirements, ensuring that research outputs from Horizon Europe, Euratom, and earlier Framework Programmes are freely accessible, thereby accelerating scientific discovery and innovation.
\nThe EU Open Research Repository serves as a complementary platform to the Open Research Europe (ORE) publishing platform. Open Research Europe focuses on providing a publishing venue for peer-reviewed articles, ensuring that research meets rigorous academic standards. The EU Open Research Repository provides a space for all the other research outputs including data sets, software, posters, and presentations that are out of scope for ORE. This holistic approach enables researchers to not only publish their findings but also share the underlying data and materials that support their work, fostering transparency and reproducibility in the scientific process.
\nThe EU Open Research Repository is funded by the European Union under grant agreement no. 101256740 (HORIZON-ZEN Plus). For more information about the project see https://about.zenodo.org/projects/horizon-zen/.
", "title": "EU Open Research Repository", "type": { "id": "organization" }, "website": "https://research-and-innovation.ec.europa.eu" }, "revision_id": 24, "slug": "eu", "theme": { "brand": "horizon", "enabled": true, "style": { "font": { "family": "Arial, sans-serif", "size": "16px", "weight": 600 }, "mainHeaderBackgroundColor": "#FFFFFF", "primaryColor": "#004494", "primaryTextColor": "#FFFFFF", "secondaryColor": "#FFD617", "secondaryTextColor": "#000000", "tertiaryColor": "#e3eefd", "tertiaryTextColor": "#1c5694" } }, "updated": "2026-03-31T12:29:43.596724+00:00" }, "revision_id": 3, "slug": "newseye", "updated": "2025-03-31T15:52:50.396736+00:00" }, { "access": { "member_policy": "open", "members_visibility": "public", "record_submission_policy": "open", "review_policy": "closed", "visibility": "public" }, "children": { "allow": false }, "created": "2019-06-01T14:55:12.024928+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "6376e1f9-b8c1-4e68-af3b-d2960ab52ecc", "links": {}, "metadata": { "description": "Impresso \u2013 Media Monitoring of the Past is an interdisciplinary project using machine learning to enable a paradigm shift in processing, enriching, representing, and exploring historical media across modalities, time periods, languages, and borders.", "funding": [ { "award": { "id": "00yjd3n13::213585" }, "funder": { "id": "00yjd3n13" } } ], "page": "The impresso project aims at enabling critical text mining of newspaper archives with the implementation of a technological framework to extract, process, link, and explore data from print media archives. The project lasts from September 2017 until August 2020 and is funded by the Swiss National Science Foundation under the Sinergia grant number CR-SII5_173719.
", "title": "Impresso - Media Monitoring of the Past", "type": { "id": "project" }, "website": "https://impresso-project.ch" }, "revision_id": 2, "slug": "impresso", "updated": "2025-07-18T10:22:23.832344+00:00" }, { "access": { "member_policy": "open", "members_visibility": "public", "record_submission_policy": "open", "review_policy": "open", "visibility": "public" }, "children": { "allow": false }, "created": "2022-02-11T13:17:33.651553+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "e054c200-d2d3-43c8-af38-c9813962ee5c", "links": {}, "metadata": { "curation_policy": "The HIPE-EVAL Zenodo community is for datasets, guidelines and papers related to the HIPE shared tasks.
\r\n", "description": "HIPE is a series of evaluation campaigns on named entity processing on historical documents in multiple languages.", "page": "HIPE is a series of evaluation campaigns on named entity processing on historical documents in multiple languages.
\r\n", "title": "HIPE evaluation campaigns" }, "revision_id": 0, "slug": "hipe-eval", "updated": "2022-02-11T13:18:06.841769+00:00" }, { "access": { "member_policy": "open", "members_visibility": "restricted", "record_submission_policy": "open", "review_policy": "closed", "visibility": "public" }, "children": { "allow": true }, "created": "2022-11-23T15:53:29.436323+00:00", "custom_fields": {}, "deletion_status": { "is_deleted": false, "status": "P" }, "id": "f0a8b890-f97a-4eb2-9eac-8b8a712d3a6c", "links": {}, "metadata": { "curation_policy": "The EU Open Research Repository serves as a repository for research outputs (data, software, posters, presentations, publications, etc) which have been funded under an EU research funding programme such as Horizon Europe, Euratom or earlier Framework Programmes.
\nThe community is managed by CERN on behalf of the European Commission.
\nZenodo’s general policies and Terms of Use apply to all content.
\nThe EU Open Research Repository accepts all digital research objects which is a research output stemming from one of EU’s research and innovation funding programmes. The funding programmes currently include:
\nHorizon Europe (including ERC, MSCA), earlier Framework Programmes (eg Horizon 2020) as well as Euratom.
\nIn line with the principle as open as possible, as closed as necessary both public and restricted content is accepted. See note on how Zenodo handles restricted content.
\nEU programme beneficiaries are eligible to submit content to the community. The community supports three types of content submissions:
\nSubmission via an EU Project Community (through user interface or programmatic APIs).
\nSubmission directly to the EU Open Research Repository.
\nAutomated harvesting from existing Zenodo content.
\nA representative of an EU project may request an EU Project Community and invite other project participants as members of the community. The project community is linked to one or more European Commission grants. All records in the project community are automatically integrated into the EU Open Research Repository immediately upon acceptance into the project community.
\nAny user may submit a record directly to the EU Open Research Repository. The submission will be moderated by Zenodo staff for compliance with the minimal required metadata requirements and its correctness.
\nRecords found among Zenodo’s existing content will on a regular basis automatically be integrated if they are found to comply with the requirements. The submissions through this method are integrated into the EU Open Research Repository with delay in a fully automated way.
\nRecords in the EU Open Research Repository are required to comply with the following minimal metadata requirements:
\nVisibility: Both public and restricted (with or without embargo and/or access request)
\nResource types: All resource types.
\nLicenses: Public and embargoed records MUST specify a license. The chosen license SHOULD be compliant with the Horizon Europe open science requirements (see Open Science in Horizon Europe)
\nFunding information: Records MUST specify at least one grant from the European Commission.
\nCreators: Creators SHOULD be identified with a persistent identifier (e.g. ORCID, GND, …), and affiliations SHOULD be identified with a persistent identifier (e.g. ROR, ISNI, …)
\nSubjects: Records SHOULD specify one or more fields of science from the European Science Vocabulary.
\nThese metadata requirements comes from the related open science requirements in Horizon Europe which are detailed in each project's grant agreement.
\nAll submissions undergo automated curation checks for compliance with the policy. Submissions through project communities are in addition reviewed by the project community. Submission directly to the EU Open Research Repository is in addiotn reviewed by Zenodo staff.
\nCommunity curators may at any point edit metadata of the records in the community without notice through human or automated processing. The curators may at their sole discretion remove records from the community that are deemed not to comply with the content and curation policy or which are deemed of insufficient quality.
\nThe content and curation policy is subject to change by the community owner at any time and without notice, other than through updating this page.
", "description": "Open repository for EU-funded research outputs from Horizon Europe, Euratom, and earlier Framework Programmes.", "funding": [ { "funder": { "id": "00k4n6c32" } } ], "organizations": [ { "id": "00k4n6c32" } ], "page": "The EU Open Research Repository is a Zenodo-community dedicated to fostering open science and enhancing the visibility and accessibility of research outputs funded by the European Union. The community is managed by CERN on behalf of the European Commission.
\nThe mission of the repository is to support the implementation of the EU's open science policy, providing a trusted and comprehensive space for researchers to share their research outputs such as data, software, reports, presentations, posters and more. The EU Open Research Repository simplifies the process of complying with open science requirements, ensuring that research outputs from Horizon Europe, Euratom, and earlier Framework Programmes are freely accessible, thereby accelerating scientific discovery and innovation.
\nThe EU Open Research Repository serves as a complementary platform to the Open Research Europe (ORE) publishing platform. Open Research Europe focuses on providing a publishing venue for peer-reviewed articles, ensuring that research meets rigorous academic standards. The EU Open Research Repository provides a space for all the other research outputs including data sets, software, posters, and presentations that are out of scope for ORE. This holistic approach enables researchers to not only publish their findings but also share the underlying data and materials that support their work, fostering transparency and reproducibility in the scientific process.
\nThe EU Open Research Repository is funded by the European Union under grant agreement no. 101256740 (HORIZON-ZEN Plus). For more information about the project see https://about.zenodo.org/projects/horizon-zen/.
", "title": "EU Open Research Repository", "type": { "id": "organization" }, "website": "https://research-and-innovation.ec.europa.eu" }, "revision_id": 24, "slug": "eu", "theme": { "brand": "horizon", "enabled": true, "style": { "font": { "family": "Arial, sans-serif", "size": "16px", "weight": 600 }, "mainHeaderBackgroundColor": "#FFFFFF", "primaryColor": "#004494", "primaryTextColor": "#FFFFFF", "secondaryColor": "#FFD617", "secondaryTextColor": "#000000", "tertiaryColor": "#e3eefd", "tertiaryTextColor": "#1c5694" } }, "updated": "2026-03-31T12:29:43.596724+00:00" } ], "ids": [ "62edd9a9-6173-4379-9408-21a4bff4269f", "6376e1f9-b8c1-4e68-af3b-d2960ab52ecc", "e054c200-d2d3-43c8-af38-c9813962ee5c", "f0a8b890-f97a-4eb2-9eac-8b8a712d3a6c" ] }, "id": "6979576", "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.6979576", "provider": "datacite" } } }, "pids": { "doi": { "client": "datacite", "identifier": "10.5281/zenodo.6979577", "provider": "datacite" }, "oai": { "identifier": "oai:zenodo.org:6979577", "provider": "oai" } }, "revision_id": 7, "stats": { "all_versions": { "data_volume": 425463040.0, "downloads": 320, "unique_downloads": 293, "unique_views": 452, "views": 472 }, "this_version": { "data_volume": 424133468.0, "downloads": 319, "unique_downloads": 292, "unique_views": 450, "views": 470 } }, "status": "published", "swh": {}, "updated": "2024-07-16T06:40:50.546911+00:00", "versions": { "index": 1, "is_latest": true } }