Report Open Access

HEP Application Delivery on HPC Resources

Shaffer, Tim; Blomer, Jakob; Ganis, Gerardo

DataCite XML Export

<?xml version='1.0' encoding='utf-8'?>
<resource xmlns:xsi="" xmlns="" xsi:schemaLocation="">
  <identifier identifierType="DOI">10.5281/zenodo.61157</identifier>
      <creatorName>Shaffer, Tim</creatorName>
      <affiliation>CERN openlab Summer Student</affiliation>
      <creatorName>Blomer, Jakob</creatorName>
      <affiliation>Summer Student Supervisor</affiliation>
      <creatorName>Ganis, Gerardo</creatorName>
      <affiliation>Summer Student Supervisor </affiliation>
    <title>HEP Application Delivery on HPC Resources</title>
    <subject>CERN openlab summer student</subject>
    <date dateType="Issued">2016-08-30</date>
  <resourceType resourceTypeGeneral="Text">Report</resourceType>
    <alternateIdentifier alternateIdentifierType="url"></alternateIdentifier>
    <relatedIdentifier relatedIdentifierType="URL" relationType="IsPartOf"></relatedIdentifier>
    <rights rightsURI="">Creative Commons Attribution 4.0 International</rights>
    <rights rightsURI="info:eu-repo/semantics/openAccess">Open Access</rights>
    <description descriptionType="Abstract">&lt;p&gt;Project Specification&lt;/p&gt;

&lt;p&gt;High-performance computing (HPC) contributes a significant and growing share of&amp;nbsp;resource to high-energy physics (HEP). Individual supercomputers such as&amp;nbsp;Edison or&amp;nbsp;Titan in the U.S. or SuperMUC in Europe deliver a raw performance of the same order of&amp;nbsp;magnitude than the Worldwide LHC Computing Grid. As we have seen with codes from&amp;nbsp;ALICE and ATLAS, it is notoriously difficult to deploy high-energy physics applications&amp;nbsp;on supercomputers, even though they often run a standard Linux on Intel x86_64 CPUs.&lt;/p&gt;

&lt;p&gt;The three main problems are:&lt;/p&gt;

&lt;p&gt;1. Limited or no Internet access;&lt;/p&gt;

&lt;p&gt;2. The lack of privileged local system rights;&lt;/p&gt;

&lt;p&gt;3. The concept of cluster submission or whole-node submission of jobs in contrast to&amp;nbsp;single CPU slot submission in HEP.&lt;/p&gt;

&lt;p&gt;Generally, the delivery of applications to hardware resources in high-energy physics is&amp;nbsp;done by CernVM-FS [1]. CernVM-FS is optimized for high-throughput resources.&amp;nbsp;Nevertheless, some successful results on HPC resources where&amp;nbsp;achieved using the Parrot&amp;nbsp;system[2] that allows to use CernVM-FS without special privileges. Building on these&amp;nbsp;results, the project aims to prototype a toolkit for application delivery that seamlessly&amp;nbsp;integrates with HEP experiments job submission systems, for instance with ALICE AliEn&amp;nbsp;or ATLAS PanDA. The&amp;nbsp;task includes a performance study of the parrot-induced&amp;nbsp;overhead which will be used to guide performance tuning for both CernVM-FS and&amp;nbsp;Parrot on typical&amp;nbsp;supercomputers. The project should further deliver a lightweight&amp;nbsp;scheduling shim that translates HEP&amp;rsquo;s job slot allocation to a whole&amp;nbsp;node or cluster-based&amp;nbsp;allocation. Finally, in order to increase the turn-around of the evaluation of new&amp;nbsp;supercomputers, a set of &amp;quot;canary jobs&amp;quot; should be&amp;nbsp;collected that validate HEP codes on&amp;nbsp;new resources.&lt;/p&gt;




&lt;p&gt;On high performance computing (HPC) resources, users have less control over&amp;nbsp;worker&amp;nbsp;nodes than in the grid. Using HPC resources for high energy physics&amp;nbsp;applications&amp;nbsp;becomes more complicated because individual nodes often&amp;nbsp;don&amp;#39;t have Internet&amp;nbsp;connectivity or a filesystem configured to use as a local&amp;nbsp;cache. The current solution in&amp;nbsp;CVMFS preloads the cache from a gateway node onto the shared cluster file system.&amp;nbsp;This approach works but does not scale&amp;nbsp;well into large production environments. In this&amp;nbsp;project, we develop an in&amp;nbsp;memory cache for CVMFS, and assess approaches to running&amp;nbsp;jobs without&amp;nbsp;special privilege on the worker nodes. We propose using Parrot and CVMFS&amp;nbsp;with RAM cache as a viable approach to HEP application delivery on&amp;nbsp;HPC resources.&lt;/p&gt;</description>
All versions This version
Views 3333
Downloads 2323
Data volume 10.4 MB10.4 MB
Unique views 3232
Unique downloads 2222


Cite as