Report Open Access

Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development

Ferreira da Silva, Rafael; Casanova, Henri; Chard, Kyle; Coleman, Tainã; Laney, Dan; Ahn, Dong; Jha, Shantenu; Howell, Dorran; Soiland-Reys, Stian; Altintas, Ilkay; Thain, Douglas; Filgueira, Rosa; Babuji, Yadu; Badia, Rosa M.; Balis, Bartosz; Caino-Lores, Silvina; Callaghan, Scott; Coppens, Frederik; Crusoe, Michael R.; De, Kaushik; Di Natale, Frank; Do, Tu M. A.; Enders, Bjoern; Fahringer, Thomas; Fouilloux, Anne; Fursin, Grigori; Gaignard, Alban; Ganose, Alex; Garijo, Daniel; Gesing, Sandra; Goble, Carole; Hasan, Adil; Huber, Sebastiaan; Katz, Daniel S.; Leser, Ulf; Lowe, Douglas; Ludaescher, Bertram; Maheshwari, Ketan; Malawski, Maciej; Mayani, Rajiv; Mehta, Kshitij; Merzky, Andre; Munson, Todd; Ozik, Jonathan; Pottier, Loïc; Ristov, Sashko; Roozmeh, Mehdi; Souza, Renan; Suter, Frédéric; Tovar, Benjamin; Turilli, Matteo; Vahi, Karan; Vidal-Torreira, Alvaro; Whitcup, Wendy; Wilde, Michael; Williams, Alan; Wolf, Matthew; Wozniak, Justin

Scientific workflows are a cornerstone of modern scientific computing, and they have underpinned some of the most significant discoveries of the last decade. Many of these workflows have high computational, storage, and/or communication demands, and thus must execute on a wide range of large-scale platforms, from large clouds to upcoming exascale HPC platforms. Workflows will play a crucial role in the data-oriented and post-Moore’s computing landscape as they democratize the application of cutting-edge research techniques, computationally intensive methods, and use of new computing platforms. As workflows continue to be adopted by scientific projects and user communities, they are becoming more complex. Workflows are increasingly composed of tasks that perform computations such as short machine learning inference, multi-node simulations, long-running machine learning model training, amongst others, and thus increasingly rely on heterogeneous architectures that include CPUs but also GPUs and accelerators. The workflow management system (WMS) technology landscape is currently segmented and presents significant barriers to entry due to the hundreds of seemingly comparable, yet incompatible, systems that exist. Another fundamental problem is that there are conflicting theoretical bases and abstractions for a WMS. Systems that use the same underlying abstractions can likely be translated between, which is not the case for systems that use different abstractions. More information: https://workflowsri.org/summits/technical

Files (16.8 MB)
Name Size
Workflows_Community_Summit_Apr_2021.pdf
md5:3e4ae3aae9ae8b0da4ad996e57f29214
16.8 MB Download
90
68
views
downloads
All versions This version
Views 9090
Downloads 6868
Data volume 1.1 GB1.1 GB
Unique views 8181
Unique downloads 6363

Share

Cite as