ResearchObject.org (RO)
Published October 20, 2023 | Version v1
Publication Open

Building the I (Interoperability) of FAIR for Performance Reproducibility of Large-Scale Composable Workflows in RECUP

Creators

  • 1. ROR icon Argonne National Laboratory
  • 2. ROR icon Texas State University
  • 3. ROR icon Brookhaven National Laboratory
  • 4. ROR icon Rochester Institute of Technology
  • 5. ROR icon Tufts University

Description

Scientific computing communities increasingly run their experiments using complex data- and compute-intensive workflows that utilize distributed and heterogeneous architectures targeting numerical simulations and machine learning, often executed on the Department of Energy Leadership Computing Facilities (LCFs).  We argue that a principled, systematic approach to implementing FAIR principles at scale, including fine-grained metadata extraction and organization, can help with the numerous challenges to performance reproducibility posed by such workflows.  We extract workflow patterns, propose a set of tools to manage the entire life cycle of performance metadata, and aggregate them in an HPC-ready framework for reproducibility (RECUP).  We describe the challenges in making these tools interoperable, preliminary work, and lessons learned from this experiment

Files

2023_escience-ReWords_workshop (1).pdf

Files (609.0 kB)

Name Size Download all
md5:4200022c93c14b74571495a02d95f85f
609.0 kB Preview Download

Additional details

Dates

Accepted
2023-09-13