There is a newer version of this record available.

Working paper Open Access

StreamFlow: cross-breeding cloud with HPC

Iacopo Colonnelli; Barbara Cantalupo; Ivan Merelli; Marco Aldinucci

Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow.

Files (552.7 kB)
Name Size
2002.01558v2.pdf
md5:fc1424cf5f3c4c01b5f6374c13ed4129
552.7 kB Download
116
91
views
downloads
All versions This version
Views 11657
Downloads 9145
Data volume 52.6 MB24.9 MB
Unique views 8744
Unique downloads 8542

Share

Cite as