Report Open Access

Performance Visualization of ROOT/IO on HPC Storage Systems

Rui Pedro Neto Reis

HPC systems are becoming ever more important as a data processing resource for the LHC experiments. HPC sites typically use storage systems different from the well-understood HEP storage systems. Current HPC sites deploy high-performance cluster file systems (Lustre, Ceph-FS, GPFS), sometimes amended by burst buffers. Next-generation exascale systems are expected to gradually move from file systems to object stores (e.g., Intel DAOS) to address storage scalability challenges. On the application side, the ROOT I/O library provides the predominant I/O base layer for LHC data handling. To optimally use so-far untapped storage systems, applications built on top of the ROOT I/O need to be tuned for the target storage system at hand (e.g., in terms of I/O block sizes, level of I/O parallelism, networking link parameters, etc).

This project aims at providing visualizations of key performance indicators of the RNTuple I/O subsystem. The existing framework for collecting performance metrics should be extended to not only keep metric aggregates but also metric histograms. Metric aggregates and histograms should be presented as several performance overview plots, e.g., request size distribution, distribution of the I/O queue depth, access pattern of the data. As a first application, the new visualizations can be used to better understand and improve the RNTuple storage backend for Intel DAOS.

Files (587.4 kB)
Name Size
CERN_openlab_SUM_report_Rui_Reis.pdf
md5:4c3a6d72f703f1ea4076b29c50caafce
587.4 kB Download
61
32
views
downloads
All versions This version
Views 6161
Downloads 3232
Data volume 18.8 MB18.8 MB
Unique views 5959
Unique downloads 3131

Share

Cite as