Working paper Open Access

Scaling of Biological Data Work ows to Large HPC Systems - A Case Study in Marine Genomics -

Thomas Röblitz

Other(s)
Ole W. Saastad; Hans A. Eide; Katerina Michalickova; Alexander Johan Nederbragt; Bastiaan Star

Sequencing projects, like the Aqua Genome project, generate vast amounts of data which is processed through dif-
ferent work ows composed of several steps linked together. Currently, such workflows are often run manually on
large servers. With the increasing amount of raw data that approach is no longer feasible. The successful imple-
mentation of the project's goals requires 2-3 orders of magnitude scaling of computing, while achieving high reli-
ability on and supporting ease-of-use of super computing resources at the same time. We describe two example
use cases, the implementation challenges and constraints, the actual application enabling and report our ndings.

Files (504.4 kB)
Name Size
WP171.pdf
md5:50508e75466ab6fbd53d3204d172b794 		504.4 kB Download
35
16
views
downloads
All versions This version
Views 3535
Downloads 1616
Data volume 8.1 MB8.1 MB
Unique views 3434
Unique downloads 1515
More info on how stats are collected.
Indexed in
Publication date:
June 4, 2014
DOI:
10.5281/zenodo.823031

Zenodo DOI Badge

DOI 

10.5281/zenodo.823031

Markdown 

[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.823031.svg)](https://doi.org/10.5281/zenodo.823031)

reStructedText 

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.823031.svg
   :target: https://doi.org/10.5281/zenodo.823031

HTML 

<a href="https://doi.org/10.5281/zenodo.823031"><img src="https://zenodo.org/badge/DOI/10.5281/zenodo.823031.svg" alt="DOI"></a>

Image URL 

https://zenodo.org/badge/DOI/10.5281/zenodo.823031.svg

Target URL 

https://doi.org/10.5281/zenodo.823031

Keyword(s):
workflows, magnitude scaling
Grants:
European Commission:
  • PRACE-3IP - PRACE - Third Implementation Phase Project (312763)
Communities:
License (for files):
Creative Commons Attribution 4.0 International

Versions

Version 1 10.5281/zenodo.823031 Jun 4, 2014
Cite all versions? You can cite all versions by using the DOI 10.5281/zenodo.823030. This DOI represents all versions, and will always resolve to the latest one. Read more.

Share

Cite as

Export