Published June 30, 2022
| Version v1
Poster
Open
An intelligent Data Delivery Service (iDDS) for and beyond the ATLAS experiment
Authors/Creators
- 1. University of Wisconsin
- 2. Universidad Andres Bello
- 3. University of Wisconsin Madison
- 4. University of Texas at Arlington
- 5. Brookhaven National Laboratory
- 6. BNL
Description
The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflow and data management systems, decoupling data pre-processing, delivery, and main processing in various workflows. It is an experiment-agnostic service that has been deployed to serve data carousel, hyperparameter optimization, multiple-steps DAG (Directed Acyclic Graphs) workflows and so on. Here we will present the motivation for iDDS, the architecture, use cases and current status for ATLAS and Rubin Observatory exercise, and plans for the future.
Files
144_idds_lp2021.mp4
Files
(12.2 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:68ea811f7ffea1efb52326b4c0680f67
|
9.9 MB | Preview Download |
|
md5:b4dff2e358ed75d3ff6dfcd4d87c167a
|
2.3 MB | Preview Download |