Project deliverable Open Access

"Implementation Notes for the Storage and Data Access Infrastructure", ExaNeSt project Deliverable D4.3

ExaNeSt Consortium


DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://datacite.org/schema/kernel-3" xsi:schemaLocation="http://datacite.org/schema/kernel-3 http://schema.datacite.org/meta/kernel-3/metadata.xsd">
  <identifier identifierType="DOI">10.5281/zenodo.802198</identifier>
  <creators>
    <creator>
      <creatorName>ExaNeSt Consortium</creatorName>
    </creator>
  </creators>
  <titles>
    <title>"Implementation Notes For The Storage And Data Access Infrastructure", Exanest Project Deliverable D4.3</title>
  </titles>
  <publisher>Zenodo</publisher>
  <publicationYear>2017</publicationYear>
  <subjects>
    <subject>European Union (EU)</subject>
    <subject>Horizon 2020</subject>
    <subject>Euratom</subject>
    <subject>Euratom research &amp; training programme 2014-2018</subject>
  </subjects>
  <contributors>
    <contributor contributorType="Other">
      <contributorName>Bernd Lietzow</contributorName>
      <affiliation>FHG</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Anastasios Papagiannis</contributorName>
      <affiliation>FORTH</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Manolis Ploumidis</contributorName>
      <affiliation>FORTH</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Manolis Marazakis</contributorName>
      <affiliation>FORTH</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Angelos Mouzakitis</contributorName>
      <affiliation>VOSYS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Alvise Rigo</contributorName>
      <affiliation>VOSYS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Lucia Morganti</contributorName>
      <affiliation>INFN</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Ying Zhang</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Panagiotis Koutsourakis</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Joeri van Ruth</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Martin Kersten</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Niels Nes</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Other">
      <contributorName>Sjoerd Mullender</contributorName>
      <affiliation>MDBS</affiliation>
    </contributor>
    <contributor contributorType="Funder">
      <contributorName>European Commission</contributorName>
      <nameIdentifier nameIdentifierScheme="info">info:eu-repo/grantAgreement/EC/H2020/671553/</nameIdentifier>
    </contributor>
  </contributors>
  <dates>
    <date dateType="Issued">2017-05-30</date>
  </dates>
  <resourceType resourceTypeGeneral="Text">Project deliverable</resourceType>
  <alternateIdentifiers>
    <alternateIdentifier alternateIdentifierType="url">https://zenodo.org/record/802198</alternateIdentifier>
  </alternateIdentifiers>
  <relatedIdentifiers>
    <relatedIdentifier relatedIdentifierType="DOI" relationType="IsPartOf">10.5281/zenodo.802197</relatedIdentifier>
  </relatedIdentifiers>
  <rightsList>
    <rights rightsURI="https://creativecommons.org/licenses/by-nc-nd/4.0/">Creative Commons Attribution-NonCommercial-NoDerivatives</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
  </rightsList>
  <descriptions>
    <description descriptionType="Abstract">&lt;p&gt;In this deliverable of WP4, D4.3 “Implementation Notes for the Storage and Data Access Infrastructure”, we describe the  implementation of each software component and tool that was specified in deliverable D4.2:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Extensions and enhancements to the common I/O path in Linux with focus on two key areas: (i) supporting protected access to storage devices from user space, i.e. direct access to storage with minimal kernel related overheads; and (ii) enhancements to the access path for memory-mapped file access by, amongst others, making use of NVM devices. [FORTH]&lt;/li&gt;
	&lt;li&gt;Extensions to the BeeGFS parallel file system: Metadata replication mechanisms to handle consistency management and resilience  to failures, as well as incorporating those mechanisms in a caching extension. [FHG]&lt;/li&gt;
	&lt;li&gt;Acceleration mechanisms for Host-to-VM and VM-to-VM interactions that take into account properties of the hardware platform and the unified interconnect. Two main technologies for hardware-assisted virtualisation are considered: (i) use of RDMA capabilities for accessing remote memory from within a virtual machine, and (ii) HPC API remoting to improve the performance of HPC APIs (specifically MPI) for applications that execute as an ensemble of virtual machines. [VOSYS]&lt;/li&gt;
	&lt;li&gt;The design of a new replication mechanism for analytical databases, and its implementation in MonetDB. This replication scheme has been designed in such a way that it will not only help improving the availability, reliability, performance and scalability of a database server in general, but also help exploiting the ExaNeSt storage infrastructure and the ExaNeSt platform in particular. [MDBS]&lt;/li&gt;
	&lt;li&gt;Several monitoring and testing tools: the ExaNeSt storage administration and monitoring [INFN]; experiment automation, stress-load and fault injection tools [FORTH]; tests simulating HPC application I/O behaviours [INFN]; and a database profiler [MDBS].&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In the storage infrastructure, resilience is mainly addressed by replication mechanisms in the file system. The system’s resilience will be stressed by the stress-load and fault injection tools, while its health is monitored by the monitoring tools. In addition, the system’s resilience level will be evaluated by the HPC and DBMS applications. To achieve the objectives of WP4, the storage and data access infrastructure proposed by ExaNeSt is built upon the following main components: (i) the distributed file system BeeGFS, (ii) Linux data access technologies, (iii) KVM based virtualisation, and (iv) storage system monitoring and administration tools. Moreover, we will use the state-of-the-art HPC applications and analytical databases to guide the design of the envisioned infrastructure and to evaluate and showcase the result.&lt;/p&gt;</description>
    <description descriptionType="Other">ExaNeSt project Deliverable D4.3</description>
  </descriptions>
</resource>

Share

Cite as