Published August 30, 2020 | Version v3
Journal article Open

StreamFlow: cross-breeding cloud with HPC

  • 1. Department of Computer Science, University of Torino, Italy
  • 2. Biomedical Technologies (ITB) of the Italian National Research Council (CNR)

Description

Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single-cell transcriptomic data analysis workflow.

Files

StreamFlowIEEE-TETC.pdf

Files (602.5 kB)

Name Size Download all
md5:4f260ffa081c9c284de9c872b80b5162
602.5 kB Preview Download

Additional details

Related works

Is derived from
Journal article: 10.1109/TETC.2020.3019202 (DOI)

Funding

DeepHealth – Deep-Learning and HPC to Boost Biomedical Applications for Health 825111
European Commission