Working paper Open Access
StreamFlow: cross-breeding cloud with HPC
Iacopo Colonnelli;
Barbara Cantalupo;
Ivan Merelli;
Marco Aldinucci
Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow.
| Name | Size | |
|---|---|---|
|
2002.01558v2.pdf
md5:fc1424cf5f3c4c01b5f6374c13ed4129 |
552.7 kB | Download |
| All versions | This version | |
|---|---|---|
| Views | 231 | 118 |
| Downloads | 145 | 70 |
| Data volume | 83.9 MB | 38.7 MB |
| Unique views | 154 | 82 |
| Unique downloads | 139 | 67 |
- Publication date:
- February 26, 2020
- DOI:
-
- Keyword(s):
- Workflow High-Performance Computing Cloud Bioinformatics
- Grants:
-
European Commission:
- DeepHealth - Deep-Learning and HPC to Boost Biomedical Applications for Health (825111)
- Communities:
- License (for files):
- Creative Commons Attribution 4.0 International