Service Incident: New DOI registrations are working again. Re-registration of failed DOI registrations (~500) are still affected by the service incident at DataCite (our DOI registration agency).
Published November 24, 2016 | Version 1.3.0
Software Open

The D-NET software toolkit: dnet-basic-aggregator

Description

D-Net Software Toolikt

The D-NET Software Toolkit is a system that offers functionalities for the collection (“harvesting”), transformation, aggregation, and indexing of metadata records collected from an arbitrary number of data sources, complying with different protocols and data exchange formats. D-NET sets a workflow language, which developers can use to combine a variety of D-NET data management services, configure them to handle data according to given data models, and pipeline them into autonomic data processing workflows.

This software package is a simplified version of the D-Net toolkit and consists of a web application with a minimal set of services for:

  • Collection of metadata records in oai_dc format via OAI-PMH, FTP, local file system, HTTP.

  • Transformation of the collected metadata records into an internal format named DMF (Driver Metadata Format)

  • Indexing of DMF records in a Solr full-text index

  • OAI-PMH export of aggregated metadata records in DMF and oai_dc formats. More formats can be added at runtime by providing a dedicated XSLT from DMF to the desired target format.

Major changes in version 1.3.0

  • OAI Publisher:
    • fixed cache management
    • fixed oai consistency (post feed) workflow branch
    • fixed deletion of content when workflow of data sources are deleted
  • D-Net enabling services:
    • using cache for subscription access
    • support only one subscription registry
  • Mongo based services (mdstore, oaistore, wf logging):
    • using API of mongo-java-driver 3.2.2, removed usage of deprecated methods
    • tracking the number of stored records to possibly highlight the collection of records with the same identifier
  • GUI:
    • enabling deletion of APIs via GUI
    • enabling editing of metadata_identifier_path
    • more info available in the datasource section
    • removed map of data sources (TODO: adapt to the new google map API)
  • Metadata collection:
    • handling HTML illegal entities in collected XMLs
  • Indexing:
    • default query operator for "bag of words" queries set to AND instead of OR
  • Workflow manager
    • do not launch workflows that were scheduled for execution during a pause of the aggregation system ("prepare for shutdown")

Official Web Site: http://www.d-net.research-infrastructures.eu

Need support? Contact us via email on dnet-team[at]isti.cnr.it

Files

dnet-team/dnet-basic-aggregator-1.3.0.zip

Files (392.9 kB)

Name Size Download all
md5:119ee0ff5f82d26be1987fc607d3df38
392.9 kB Preview Download

Additional details

Funding

OpenAIRE-Advance – OpenAIRE Advancing Open Scholarship 777541
European Commission
OPENAIRE – Open Access Infrastructure for Research in Europe 246686
European Commission
OpenAIRE-Connect – OpenAIRE - CONNECTing scientific results in support of Open Science 731011
European Commission
PARTHENOS – Pooling Activities, Resources and Tools for Heritage E-research Networking, Optimization and Synergies 654119
European Commission
OpenAIRE2020 – Open Access Infrastructure for Research in Europe 2020 643410
European Commission
OPENAIREPLUS – 2nd-Generation Open Access Infrastructure for Research in Europe 283595
European Commission
ARIADNE – Advanced Research Infrastructure for Archaeological Dataset Networking in Europe 313193
European Commission