Software Open Access
The D-NET software toolkit: dnet-basic-aggregator
Michele Artini;
Claudio Atzori;
Alessia Bardi;
Sandro La Bruzzo;
Paolo Manghi;
Andrea Mannocci
MARC21 XML Export
<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
<leader>00000nmm##2200000uu#4500</leader>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">aggregator</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">soa</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">soap</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">oai_pmh</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">oai</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">dnet</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">metadata</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">distributed systems</subfield>
</datafield>
<datafield tag="653" ind1=" " ind2=" ">
<subfield code="a">digital library</subfield>
</datafield>
<controlfield tag="005">20201202171659.0</controlfield>
<controlfield tag="001">168362</controlfield>
<datafield tag="700" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0001-9613-6639</subfield>
<subfield code="a">Claudio Atzori</subfield>
</datafield>
<datafield tag="700" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0002-1112-1292</subfield>
<subfield code="a">Alessia Bardi</subfield>
</datafield>
<datafield tag="700" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0003-2855-1245</subfield>
<subfield code="a">Sandro La Bruzzo</subfield>
</datafield>
<datafield tag="700" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0001-7291-3210</subfield>
<subfield code="a">Paolo Manghi</subfield>
</datafield>
<datafield tag="700" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0002-5193-7851</subfield>
<subfield code="a">Andrea Mannocci</subfield>
</datafield>
<datafield tag="856" ind1="4" ind2=" ">
<subfield code="s">392890</subfield>
<subfield code="z">md5:119ee0ff5f82d26be1987fc607d3df38</subfield>
<subfield code="u">https://zenodo.org/record/168362/files/dnet-team/dnet-basic-aggregator-1.3.0.zip</subfield>
</datafield>
<datafield tag="542" ind1=" " ind2=" ">
<subfield code="l">open</subfield>
</datafield>
<datafield tag="260" ind1=" " ind2=" ">
<subfield code="c">2016-11-24</subfield>
</datafield>
<datafield tag="909" ind1="C" ind2="O">
<subfield code="p">software</subfield>
<subfield code="o">oai:zenodo.org:168362</subfield>
</datafield>
<datafield tag="100" ind1=" " ind2=" ">
<subfield code="u">ISTI-CNR</subfield>
<subfield code="0">(orcid)0000-0002-4406-428X</subfield>
<subfield code="a">Michele Artini</subfield>
</datafield>
<datafield tag="245" ind1=" " ind2=" ">
<subfield code="a">The D-NET software toolkit: dnet-basic-aggregator</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">777541</subfield>
<subfield code="a">OpenAIRE Advancing Open Scholarship</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">246686</subfield>
<subfield code="a">Open Access Infrastructure for Research in Europe</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">731011</subfield>
<subfield code="a">OpenAIRE - CONNECTing scientific results in support of Open Science</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">654119</subfield>
<subfield code="a">Pooling Activities, Resources and Tools for Heritage E-research Networking, Optimization and Synergies</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">643410</subfield>
<subfield code="a">Open Access Infrastructure for Research in Europe 2020</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">283595</subfield>
<subfield code="a">2nd-Generation Open Access Infrastructure for Research in Europe</subfield>
</datafield>
<datafield tag="536" ind1=" " ind2=" ">
<subfield code="c">313193</subfield>
<subfield code="a">Advanced Research Infrastructure for Archaeological Dataset Networking in Europe</subfield>
</datafield>
<datafield tag="540" ind1=" " ind2=" ">
<subfield code="u">https://opensource.org/licenses/AGPL-3.0</subfield>
<subfield code="a">GNU Affero General Public License v3.0 or later</subfield>
</datafield>
<datafield tag="650" ind1="1" ind2="7">
<subfield code="a">cc-by</subfield>
<subfield code="2">opendefinition.org</subfield>
</datafield>
<datafield tag="520" ind1=" " ind2=" ">
<subfield code="a"><p>D-Net Software Toolikt</p>
<p>The D-NET Software Toolkit is a system that offers functionalities for the collection (&ldquo;harvesting&rdquo;), transformation, aggregation, and indexing of metadata records collected from an arbitrary number of data sources, complying with different protocols and data exchange formats. D-NET sets a workflow language, which developers can use to combine a variety of D-NET data management services, configure them to handle data according to given data models, and pipeline them into autonomic data processing workflows.</p>
<p>This software package is a simplified version of the D-Net toolkit and consists of a web application with a minimal set of services for:</p>
<ul>
<li>
<p>Collection of metadata records in oai_dc format via OAI-PMH, FTP, local file system, HTTP.</p>
</li>
<li>
<p>Transformation of the collected metadata records into an internal format named DMF (Driver Metadata Format)</p>
</li>
<li>
<p>Indexing of DMF records in a Solr full-text index</p>
</li>
<li>
<p>OAI-PMH export of aggregated metadata records in DMF and oai_dc formats. More formats can be added at runtime by providing a dedicated XSLT from DMF to the desired target format.</p>
</li>
</ul>
<p>Major changes in version&nbsp;1.3.0</p>
<ul>
<li>OAI Publisher:
<ul>
<li>fixed cache management</li>
<li>fixed oai consistency (post feed) workflow branch</li>
<li>fixed deletion of content when workflow of data sources are deleted</li>
</ul>
</li>
<li>D-Net enabling services:
<ul>
<li>using cache for subscription access</li>
<li>support only one subscription registry</li>
</ul>
</li>
<li>Mongo based services (mdstore, oaistore, wf logging):
<ul>
<li>using API of mongo-java-driver 3.2.2, removed usage of deprecated methods</li>
<li>tracking the number of stored records to possibly highlight the collection of records with the same identifier</li>
</ul>
</li>
<li>GUI:
<ul>
<li>enabling deletion of APIs via GUI</li>
<li>enabling editing of metadata_identifier_path</li>
<li>more info available in the datasource section</li>
<li>removed map of data sources (TODO: adapt to the new google map API)</li>
</ul>
</li>
<li>Metadata collection:
<ul>
<li>handling HTML illegal entities in collected XMLs</li>
</ul>
</li>
<li>Indexing:
<ul>
<li>default query operator for &quot;bag of words&quot; queries set to AND instead of OR</li>
</ul>
</li>
<li>Workflow manager
<ul>
<li>do not launch workflows that were scheduled for execution during a pause of the aggregation system (&quot;prepare for shutdown&quot;)</li>
</ul>
</li>
</ul>
<p>Official Web Site: http://www.d-net.research-infrastructures.eu</p>
<p>Need support? Contact us via email on&nbsp;dnet-team[at]isti.cnr.it</p></subfield>
</datafield>
<datafield tag="773" ind1=" " ind2=" ">
<subfield code="n">url</subfield>
<subfield code="i">isDocumentedBy</subfield>
<subfield code="a">https://github.com/dnet-team/dnet-basic-aggregator/tree/1.3.0</subfield>
</datafield>
<datafield tag="773" ind1=" " ind2=" ">
<subfield code="n">url</subfield>
<subfield code="i">isCompiledBy</subfield>
<subfield code="a">http://svn-public.driver.research-infrastructures.eu/driver/dnet45/modules</subfield>
</datafield>
<datafield tag="773" ind1=" " ind2=" ">
<subfield code="n">doi</subfield>
<subfield code="i">references</subfield>
<subfield code="a">10.1108/PROG-08-2013-0045</subfield>
</datafield>
<datafield tag="773" ind1=" " ind2=" ">
<subfield code="n">doi</subfield>
<subfield code="i">isVersionOf</subfield>
<subfield code="a">10.5281/zenodo.594239</subfield>
</datafield>
<datafield tag="024" ind1=" " ind2=" ">
<subfield code="a">10.5281/zenodo.168362</subfield>
<subfield code="2">doi</subfield>
</datafield>
<datafield tag="980" ind1=" " ind2=" ">
<subfield code="a">software</subfield>
</datafield>
</record>
818
80
views
downloads
| All versions | This version | |
|---|---|---|
| Views | 818 | 151 |
| Downloads | 80 | 19 |
| Data volume | 474.0 MB | 7.5 MB |
| Unique views | 690 | 133 |
| Unique downloads | 66 | 16 |
Indexed in
- Publication date:
- November 24, 2016
- DOI:
-
- Keyword(s):
- aggregator soa soap oai_pmh oai dnet metadata distributed systems digital library
- Grants:
-
European Commission:
-
OpenAIRE-Advance - OpenAIRE Advancing Open Scholarship (777541)
-
OPENAIRE - Open Access Infrastructure for Research in Europe (246686)
-
OpenAIRE-Connect - OpenAIRE - CONNECTing scientific results in support of Open Science (731011)
-
PARTHENOS - Pooling Activities, Resources and Tools for Heritage E-research Networking, Optimization and Synergies (654119)
-
OpenAIRE2020 - Open Access Infrastructure for Research in Europe 2020 (643410)
-
OPENAIREPLUS - 2nd-Generation Open Access Infrastructure for Research in Europe (283595)
- ARIADNE - Advanced Research Infrastructure for Archaeological Dataset Networking in Europe (313193)
-
OpenAIRE-Advance - OpenAIRE Advancing Open Scholarship (777541)
- Related identifiers:
- Compiled by
- Documented by
- References
- 10.1108/PROG-08-2013-0045 (Journal article)
- License (for files):
- GNU Affero General Public License v3.0 or later