Published December 15, 2021 | Version v1
Conference paper Open

A Survey of Big Data Pipeline Orchestration Tools from the Perspective of the DataCloud Pro ject

Authors/Creators

  • 1. KTH Royal Institute of Technology
  • 2. SINTEF AS

Description

This paper presents a survey of existing tools for Big Data pipeline orchestration based on a comparative framework developed in the DataCloud project. We propose criteria for evaluating the tools to support reusability, flexible pipeline communication modes, and separa- tion of concerns in Big Data pipeline descriptions. This survey aims to identify research and technological gaps and to recommend approaches for filling them. Further work in the DataCloud project is oriented to- wards the design, implementation, and practical evaluation of the rec- ommended approaches.

Files

DAMDID-CEUR.pdf

Files (1.6 MB)

Name Size
md5:e1cb541c4b655e19b40d258a1edaba04
1.6 MB Preview Download

Additional details

Funding

European Commission
DataCloud - ENABLING THE BIG DATA PIPELINE LIFECYCLE ON THE COMPUTING CONTINUUM 101016835