Working paper Open Access

Scaling of Biological Data Work ows to Large HPC Systems - A Case Study in Marine Genomics -

Thomas Röblitz


MARC21 XML Export

<?xml version='1.0' encoding='UTF-8'?>
<record xmlns="http://www.loc.gov/MARC21/slim">
  <leader>00000nam##2200000uu#4500</leader>
  <datafield tag="653" ind1=" " ind2=" ">
    <subfield code="a">workflows, magnitude scaling</subfield>
  </datafield>
  <controlfield tag="005">20200120120629.0</controlfield>
  <controlfield tag="001">823031</controlfield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Department for Research Computing, University Center for Information Technology (USIT), University of Oslo, P.O. Box 1059, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Ole W. Saastad</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Department for Research Computing, University Center for Information Technology (USIT), University of Oslo, P.O. Box 1059, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Hans A. Eide</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Department for Research Computing, University Center for Information Technology (USIT), University of Oslo, P.O. Box 1059, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Katerina Michalickova</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Center for Ecological and Evolutionary Synthesis, Department of Biosciences (CEES), University of Oslo, P.O. Box 1066, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Alexander Johan Nederbragt</subfield>
  </datafield>
  <datafield tag="700" ind1=" " ind2=" ">
    <subfield code="u">Center for Ecological and Evolutionary Synthesis, Department of Biosciences (CEES), University of Oslo, P.O. Box 1066, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="4">oth</subfield>
    <subfield code="a">Bastiaan Star</subfield>
  </datafield>
  <datafield tag="856" ind1="4" ind2=" ">
    <subfield code="s">504388</subfield>
    <subfield code="z">md5:50508e75466ab6fbd53d3204d172b794</subfield>
    <subfield code="u">https://zenodo.org/record/823031/files/WP171.pdf</subfield>
  </datafield>
  <datafield tag="542" ind1=" " ind2=" ">
    <subfield code="l">open</subfield>
  </datafield>
  <datafield tag="260" ind1=" " ind2=" ">
    <subfield code="c">2014-06-04</subfield>
  </datafield>
  <datafield tag="909" ind1="C" ind2="O">
    <subfield code="p">openaire</subfield>
    <subfield code="p">user-prace</subfield>
    <subfield code="o">oai:zenodo.org:823031</subfield>
  </datafield>
  <datafield tag="100" ind1=" " ind2=" ">
    <subfield code="u">Department for Research Computing, University Center for Information Technology (USIT), University of Oslo, P.O. Box 1059, Blindern, 0316 Oslo, Norway</subfield>
    <subfield code="a">Thomas Röblitz</subfield>
  </datafield>
  <datafield tag="245" ind1=" " ind2=" ">
    <subfield code="a">Scaling of Biological Data Work ows to Large HPC Systems - A Case Study in Marine Genomics -</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">user-prace</subfield>
  </datafield>
  <datafield tag="536" ind1=" " ind2=" ">
    <subfield code="c">312763</subfield>
    <subfield code="a">PRACE - Third Implementation Phase Project</subfield>
  </datafield>
  <datafield tag="540" ind1=" " ind2=" ">
    <subfield code="u">https://creativecommons.org/licenses/by/4.0/legalcode</subfield>
    <subfield code="a">Creative Commons Attribution 4.0 International</subfield>
  </datafield>
  <datafield tag="650" ind1="1" ind2="7">
    <subfield code="a">cc-by</subfield>
    <subfield code="2">opendefinition.org</subfield>
  </datafield>
  <datafield tag="520" ind1=" " ind2=" ">
    <subfield code="a">&lt;p&gt;Sequencing projects, like the Aqua Genome project, generate vast amounts of data which is processed through dif-&lt;br&gt;
ferent work ows composed of several steps linked together. Currently, such workflows are often run manually on&lt;br&gt;
large servers. With the increasing amount of raw data that approach is no longer feasible. The successful imple-&lt;br&gt;
mentation of the project's goals requires 2-3 orders of magnitude scaling of computing, while achieving high reli-&lt;br&gt;
ability on and supporting ease-of-use of super computing resources at the same time. We describe two example&lt;br&gt;
use cases, the implementation challenges and constraints, the actual application enabling and report our ndings.&lt;/p&gt;</subfield>
  </datafield>
  <datafield tag="773" ind1=" " ind2=" ">
    <subfield code="n">doi</subfield>
    <subfield code="i">isVersionOf</subfield>
    <subfield code="a">10.5281/zenodo.823030</subfield>
  </datafield>
  <datafield tag="024" ind1=" " ind2=" ">
    <subfield code="a">10.5281/zenodo.823031</subfield>
    <subfield code="2">doi</subfield>
  </datafield>
  <datafield tag="980" ind1=" " ind2=" ">
    <subfield code="a">publication</subfield>
    <subfield code="b">workingpaper</subfield>
  </datafield>
</record>
25
10
views
downloads
All versions This version
Views 2525
Downloads 1010
Data volume 5.0 MB5.0 MB
Unique views 2424
Unique downloads 99

Share

Cite as