Report Open Access

HEP Application Delivery on HPC Resources

Shaffer, Tim; Blomer, Jakob; Ganis, Gerardo

MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="">
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">CERN openlab summer student</subfield>
  <controlfield tag="005">20200120131451.0</controlfield>
  <controlfield tag="001">61157</controlfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Summer Student Supervisor</subfield>
    <subfield code="a">Blomer, Jakob</subfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Summer Student Supervisor </subfield>
    <subfield code="a">Ganis, Gerardo</subfield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">453520</subfield>
    <subfield code="z">md5:05c79632e65bd95b006fb87742e3049e</subfield>
    <subfield code="u"></subfield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2016-08-30</subfield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="p">user-cernopenlab</subfield>
    <subfield code="o"></subfield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="u">CERN openlab Summer Student</subfield>
    <subfield code="a">Shaffer, Tim</subfield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">HEP Application Delivery on HPC Resources</subfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-cernopenlab</subfield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u"></subfield>
    <subfield code="a">Creative Commons Attribution 4.0 International</subfield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2"></subfield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;Project Specification&lt;/p&gt;

&lt;p&gt;High-performance computing (HPC) contributes a significant and growing share of&amp;nbsp;resource to high-energy physics (HEP). Individual supercomputers such as&amp;nbsp;Edison or&amp;nbsp;Titan in the U.S. or SuperMUC in Europe deliver a raw performance of the same order of&amp;nbsp;magnitude than the Worldwide LHC Computing Grid. As we have seen with codes from&amp;nbsp;ALICE and ATLAS, it is notoriously difficult to deploy high-energy physics applications&amp;nbsp;on supercomputers, even though they often run a standard Linux on Intel x86_64 CPUs.&lt;/p&gt;

&lt;p&gt;The three main problems are:&lt;/p&gt;

&lt;p&gt;1. Limited or no Internet access;&lt;/p&gt;

&lt;p&gt;2. The lack of privileged local system rights;&lt;/p&gt;

&lt;p&gt;3. The concept of cluster submission or whole-node submission of jobs in contrast to&amp;nbsp;single CPU slot submission in HEP.&lt;/p&gt;

&lt;p&gt;Generally, the delivery of applications to hardware resources in high-energy physics is&amp;nbsp;done by CernVM-FS [1]. CernVM-FS is optimized for high-throughput resources.&amp;nbsp;Nevertheless, some successful results on HPC resources where&amp;nbsp;achieved using the Parrot&amp;nbsp;system[2] that allows to use CernVM-FS without special privileges. Building on these&amp;nbsp;results, the project aims to prototype a toolkit for application delivery that seamlessly&amp;nbsp;integrates with HEP experiments job submission systems, for instance with ALICE AliEn&amp;nbsp;or ATLAS PanDA. The&amp;nbsp;task includes a performance study of the parrot-induced&amp;nbsp;overhead which will be used to guide performance tuning for both CernVM-FS and&amp;nbsp;Parrot on typical&amp;nbsp;supercomputers. The project should further deliver a lightweight&amp;nbsp;scheduling shim that translates HEP&amp;rsquo;s job slot allocation to a whole&amp;nbsp;node or cluster-based&amp;nbsp;allocation. Finally, in order to increase the turn-around of the evaluation of new&amp;nbsp;supercomputers, a set of &amp;quot;canary jobs&amp;quot; should be&amp;nbsp;collected that validate HEP codes on&amp;nbsp;new resources.&lt;/p&gt;




&lt;p&gt;On high performance computing (HPC) resources, users have less control over&amp;nbsp;worker&amp;nbsp;nodes than in the grid. Using HPC resources for high energy physics&amp;nbsp;applications&amp;nbsp;becomes more complicated because individual nodes often&amp;nbsp;don&amp;#39;t have Internet&amp;nbsp;connectivity or a filesystem configured to use as a local&amp;nbsp;cache. The current solution in&amp;nbsp;CVMFS preloads the cache from a gateway node onto the shared cluster file system.&amp;nbsp;This approach works but does not scale&amp;nbsp;well into large production environments. In this&amp;nbsp;project, we develop an in&amp;nbsp;memory cache for CVMFS, and assess approaches to running&amp;nbsp;jobs without&amp;nbsp;special privilege on the worker nodes. We propose using Parrot and CVMFS&amp;nbsp;with RAM cache as a viable approach to HEP application delivery on&amp;nbsp;HPC resources.&lt;/p&gt;</subfield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.61157</subfield>
    <subfield code="2">doi</subfield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">report</subfield>
All versions This version
Views 4545
Downloads 3939
Data volume 17.7 MB17.7 MB
Unique views 4343
Unique downloads 3838


Cite as