Project deliverable Open Access

"Implementation Notes for the Storage and Data Access Infrastructure", ExaNeSt project Deliverable D4.3

ExaNeSt Consortium


MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nam##2200000uu#4500</leader>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">European Union (EU)</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Horizon 2020</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Euratom</subfield>
  </datafield>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">Euratom research &amp; training programme 2014-2018</subfield>
  </datafield>
  <controlfield tag="005">20170908073445.0</controlfield>
  <datafield tag="500" ind1=" " ind2=" ">
    <subfield code="a">ExaNeSt project Deliverable D4.3</subfield>
  </datafield>
  <controlfield tag="001">802198</controlfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">FHG</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Bernd Lietzow</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">FORTH</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Anastasios Papagiannis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">FORTH</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Manolis Ploumidis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">FORTH</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Manolis Marazakis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">VOSYS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Angelos Mouzakitis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">VOSYS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Alvise Rigo</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">INFN</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Lucia Morganti</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Ying Zhang</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Panagiotis Koutsourakis</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Joeri van Ruth</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Martin Kersten</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Niels Nes</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">MDBS</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Sjoerd Mullender</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">6030414</subfield>
    <subfield code="z">md5:d62641a5758bc9ccdecb517e10ebde0e</subfield>
    <subfield code="u">https://zenodo.org/record/802198/files/ExaNeSt_D43_v10_storageImpl.pdf</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2017-05-30</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="p">user-ecfunded</subfield>
    <subfield code="o">oai:zenodo.org:802198</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="a">ExaNeSt Consortium</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">"Implementation Notes for the Storage and Data Access Infrastructure", ExaNeSt project Deliverable D4.3</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-ecfunded</subfield>
  </datafield>
  <datafield tag="536" ind1=" " ind2=" ">
    <subfield code="c">671553</subfield>
    <subfield code="a">European Exascale System Interconnect and Storage</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u">https://creativecommons.org/licenses/by-nc-nd/4.0/</subfield>
    <subfield code="a">Creative Commons Attribution-NonCommercial-NoDerivatives</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;In this deliverable of WP4, D4.3 “Implementation Notes for the Storage and Data Access Infrastructure”, we describe the  implementation of each software component and tool that was specified in deliverable D4.2:&lt;/p&gt;

&lt;ul&gt;
	&lt;li&gt;Extensions and enhancements to the common I/O path in Linux with focus on two key areas: (i) supporting protected access to storage devices from user space, i.e. direct access to storage with minimal kernel related overheads; and (ii) enhancements to the access path for memory-mapped file access by, amongst others, making use of NVM devices. [FORTH]&lt;/li&gt;
	&lt;li&gt;Extensions to the BeeGFS parallel file system: Metadata replication mechanisms to handle consistency management and resilience  to failures, as well as incorporating those mechanisms in a caching extension. [FHG]&lt;/li&gt;
	&lt;li&gt;Acceleration mechanisms for Host-to-VM and VM-to-VM interactions that take into account properties of the hardware platform and the unified interconnect. Two main technologies for hardware-assisted virtualisation are considered: (i) use of RDMA capabilities for accessing remote memory from within a virtual machine, and (ii) HPC API remoting to improve the performance of HPC APIs (specifically MPI) for applications that execute as an ensemble of virtual machines. [VOSYS]&lt;/li&gt;
	&lt;li&gt;The design of a new replication mechanism for analytical databases, and its implementation in MonetDB. This replication scheme has been designed in such a way that it will not only help improving the availability, reliability, performance and scalability of a database server in general, but also help exploiting the ExaNeSt storage infrastructure and the ExaNeSt platform in particular. [MDBS]&lt;/li&gt;
	&lt;li&gt;Several monitoring and testing tools: the ExaNeSt storage administration and monitoring [INFN]; experiment automation, stress-load and fault injection tools [FORTH]; tests simulating HPC application I/O behaviours [INFN]; and a database profiler [MDBS].&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In the storage infrastructure, resilience is mainly addressed by replication mechanisms in the file system. The system’s resilience will be stressed by the stress-load and fault injection tools, while its health is monitored by the monitoring tools. In addition, the system’s resilience level will be evaluated by the HPC and DBMS applications. To achieve the objectives of WP4, the storage and data access infrastructure proposed by ExaNeSt is built upon the following main components: (i) the distributed file system BeeGFS, (ii) Linux data access technologies, (iii) KVM based virtualisation, and (iv) storage system monitoring and administration tools. Moreover, we will use the state-of-the-art HPC applications and analytical databases to guide the design of the envisioned infrastructure and to evaluate and showcase the result.&lt;/p&gt;</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">doi</subfield>
    <subfield code="i">isPartOf</subfield>
    <subfield code="a">10.5281/zenodo.802197</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.802198</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">deliverable</subfield>
  </datafield>
</record>

Share

Cite as