Software Open Access

The D-NET software toolkit: dnet-basic-aggregator

Michele Artini; Claudio Atzori; Alessia Bardi; Sandro La Bruzzo; Paolo Manghi; Andrea Mannocci


Citation Style Language JSON Export

{
  "publisher": "Zenodo", 
  "DOI": "10.5281/zenodo.168362", 
  "title": "The D-NET software toolkit: dnet-basic-aggregator", 
  "issued": {
    "date-parts": [
      [
        2016, 
        11, 
        24
      ]
    ]
  }, 
  "abstract": "<p>D-Net Software Toolikt</p>\n\n<p>The D-NET Software Toolkit is a system that offers functionalities for the collection (&ldquo;harvesting&rdquo;), transformation, aggregation, and indexing of metadata records collected from an arbitrary number of data sources, complying with different protocols and data exchange formats. D-NET sets a workflow language, which developers can use to combine a variety of D-NET data management services, configure them to handle data according to given data models, and pipeline them into autonomic data processing workflows.</p>\n\n<p>This software package is a simplified version of the D-Net toolkit and consists of a web application with a minimal set of services for:</p>\n\n<ul>\n\t<li>\n\t<p>Collection of metadata records in oai_dc format via OAI-PMH, FTP, local file system, HTTP.</p>\n\t</li>\n\t<li>\n\t<p>Transformation of the collected metadata records into an internal format named DMF (Driver Metadata Format)</p>\n\t</li>\n\t<li>\n\t<p>Indexing of DMF records in a Solr full-text index</p>\n\t</li>\n\t<li>\n\t<p>OAI-PMH export of aggregated metadata records in DMF and oai_dc formats. More formats can be added at runtime by providing a dedicated XSLT from DMF to the desired target format.</p>\n\t</li>\n</ul>\n\n<p>Major changes in version&nbsp;1.3.0</p>\n\n<ul>\n\t<li>OAI Publisher:\n\t<ul>\n\t\t<li>fixed cache management</li>\n\t\t<li>fixed oai consistency (post feed) workflow branch</li>\n\t\t<li>fixed deletion of content when workflow of data sources are deleted</li>\n\t</ul>\n\t</li>\n\t<li>D-Net enabling services:\n\t<ul>\n\t\t<li>using cache for subscription access</li>\n\t\t<li>support only one subscription registry</li>\n\t</ul>\n\t</li>\n\t<li>Mongo based services (mdstore, oaistore, wf logging):\n\t<ul>\n\t\t<li>using API of mongo-java-driver 3.2.2, removed usage of deprecated methods</li>\n\t\t<li>tracking the number of stored records to possibly highlight the collection of records with the same identifier</li>\n\t</ul>\n\t</li>\n\t<li>GUI:\n\t<ul>\n\t\t<li>enabling deletion of APIs via GUI</li>\n\t\t<li>enabling editing of metadata_identifier_path</li>\n\t\t<li>more info available in the datasource section</li>\n\t\t<li>removed map of data sources (TODO: adapt to the new google map API)</li>\n\t</ul>\n\t</li>\n\t<li>Metadata collection:\n\t<ul>\n\t\t<li>handling HTML illegal entities in collected XMLs</li>\n\t</ul>\n\t</li>\n\t<li>Indexing:\n\t<ul>\n\t\t<li>default query operator for &quot;bag of words&quot; queries set to AND instead of OR</li>\n\t</ul>\n\t</li>\n\t<li>Workflow manager\n\t<ul>\n\t\t<li>do not launch workflows that were scheduled for execution during a pause of the aggregation system (&quot;prepare for shutdown&quot;)</li>\n\t</ul>\n\t</li>\n</ul>\n\n<p>Official Web Site: http://www.d-net.research-infrastructures.eu</p>\n\n<p>Need support? Contact us via email on&nbsp;dnet-team[at]isti.cnr.it</p>", 
  "author": [
    {
      "family": "Michele Artini"
    }, 
    {
      "family": "Claudio Atzori"
    }, 
    {
      "family": "Alessia Bardi"
    }, 
    {
      "family": "Sandro La Bruzzo"
    }, 
    {
      "family": "Paolo Manghi"
    }, 
    {
      "family": "Andrea Mannocci"
    }
  ], 
  "version": "1.3.0", 
  "type": "article", 
  "id": "168362"
}
818
80
views
downloads
All versions This version
Views 818151
Downloads 8019
Data volume 474.0 MB7.5 MB
Unique views 690133
Unique downloads 6616

Share

Cite as