TRIPLE Deliverable: D2.5 - Report on Data Enrichment
- 1. Net7
In this deliverable, the strategies for data enrichment in TRIPLE are presented. Through the Core Pipeline, named SCRE, metadata regarding publications and projects for the Social Sciences and Humanities are automatically harvested, mapped in the TRIPLE data model, curated, enriched and finally saved in the GoTriple platform’s indexes.
The document starts by presenting the ways SCRE imports publications metadata from OAI-PMH endpoints, OpenAIRE and Isidore data dumps. This reflects the strategies for integrating content which was planned in the project. On the one hand, OAI-PMH is a
well-known and established standard for content harvesting: many data providers, especially those of small dimension, support it, facilitating therefore their onboarding in GoTriple. The support for OpenAIRE and Isidore, on the other hand, responds to the wish to also harvest data from large aggregators, a strategy that allowed GoTriple to quickly present a significant amount
of publications in its index (more than 4 million at the time of writing).
Then the normalisation strategies applied to the acquired metadata are described. By analysing the first batches of acquired data, it has been decided to define the rules to normalise and clean the attributes for the following metadata: publication date, language codes, keywords, document types, licences, access rights and authors’ names. In the document, the definition of
controlled vocabularies for some of these attributes is also presented.
Then enrichment services are explained, including language recognition, translation, automatic classification and annotation.
The services to detect duplicate publications and to disambiguate authors are also discussed, followed by the presentation of the acquisition and processing of project metadata
Some final remarks on the data enrichment process, including the difficulties that have been
faced and solved, conclude the document.