Published January 31, 2023 | Version 1.0.0

M100 dataset: time-aggregated data for anomaly detection

Description

This entry is a part of a larger data set collected from the most recent Tier-0 supercomputer hosted at CINECA (Marconi100, https://www.hpc.cineca.it/hardware/marconi100). The data covers the entirety of the system, ranging from the computing nodes (980+ computing nodes) internal information such as core loads, temperatures, frequencies, memory write/read operations, CPU power consumption, fan speed, GPU usage details, etc., to the system-wide information, including the liquid cooling infrastructure, the air conditioning system, the power supply units, workload manager statistics, and job-related information, system status alerts, and weather forecast.   
It comprises hundreds of metrics measured on each computing node, in addition to hundreds of other metrics gathered from sensors monitored along all system components.

This particular dataset is made for anomaly detection purposes, it contains the same data as the main dataset but aggregated over time, with one Parquet file for each node. The data is distributed in tarballs, each one including all the files relative to the nodes contained in a given rack. For each file, the rows represent periods of 15 minutes, with the columns being aggregated values (average, standard deviation, min, max) over all the IPMI metrics that are available for the node; an additional column contains anomaly labels from Nagios.

More details can be found in the companion repository: https://gitlab.com/ecs-lab/exadata, including the spatial distribution of the nodes in the room.

Files

Files (24.8 GB)

Name Size
md5:ae8a04b847562dfa807863100dc6777d
269.8 MB Download
md5:6313d691ce2e280bb363681983b2decf
509.1 MB Download
md5:05f4aebe4eb932ed8b2c4fd5bf65b10e
402.9 MB Download
md5:4adfa8f9c7af78401cd86974ffcc3cb0
539.5 MB Download
md5:b1b49b21769a02e3bbcc66a3627594fd
551.8 MB Download
md5:f7f057e084982f512d14ac5185b1b76a
575.4 MB Download
md5:b50cd1140eb44c17dbf7fa363d2c3659
594.1 MB Download
md5:66ad9ff2afbff27a557c5a82af80ddfb
489.3 MB Download
md5:b0cd1d24a33cb2e13c7d4b4ff1651549
519.3 MB Download
md5:8c06f6ea698fe13cf96dffb43487998e
509.9 MB Download
md5:0ba004a57e01a37123bbb2a70f2f68c0
563.0 MB Download
md5:b0c1c6447ff6689955624a8a10f4e116
536.7 MB Download
md5:81b29652d655f09715ef3429990916a8
518.2 MB Download
md5:09796154b8da2ea525fd452c5653c267
491.5 MB Download
md5:d68d9fb06ec35c47c9cfd182842dcc84
482.5 MB Download
md5:be05e8e011bf806eac0f6859387040ef
448.3 MB Download
md5:ba1dd79fbeea08f15d0afffbe7e773d3
444.7 MB Download
md5:7acce9bbdf64092fe9d5796fbc7d1167
450.2 MB Download
md5:578ebfeed6d651c07f3eb2b227479db0
535.9 MB Download
md5:c9ab8bd92fe9843a12cb4fa4fffa6861
455.1 MB Download
md5:9de6a827bfeff99b36a9e7675272fa93
514.0 MB Download
md5:39072272f9c6218ea40d122fbf0e5da1
584.8 MB Download
md5:a3eace84e42c8cce0364b925db3e1833
522.5 MB Download
md5:838af5a2775b753fb964f06376e921fd
449.1 MB Download
md5:d9796a0e9d07d6a8374625e686b3075d
535.4 MB Download
md5:ad0e112012e12b40c0573276f2caa25b
436.5 MB Download
md5:4d7ade9807695913d4154d88f5b94718
589.8 MB Download
md5:7def9a7d0468b7aca69068de7f01467d
442.1 MB Download
md5:03b96b3840f3e5e44715bb5189365d4d
579.3 MB Download
md5:e747879087a800683c9224b16f19d80a
540.3 MB Download
md5:fbe74ec00b176004fd9ce2876c043112
520.3 MB Download
md5:357f8d854509b06b21333f9f71526761
435.8 MB Download
md5:de38f2f4df6ee485c13a49c643ea13dd
545.9 MB Download
md5:9027a7550e249fe146c2dc6a1c9eaa27
526.0 MB Download
md5:311c9dd7bcb2adfb25a516306da58371
491.4 MB Download
md5:34679a04d8c5850014f2dbe97c406431
494.3 MB Download
md5:80e82123d80916020c984df6c8d7e78e
584.4 MB Download
md5:474580e0a2634c800cc8d71fd22afd76
538.5 MB Download
md5:9435eed9c0fecd0f4fb2971c06a8c89b
500.4 MB Download
md5:e212206091bf61562593d2d2e01d1c04
591.5 MB Download
md5:7c6233a5041c4a87acdd63a743fd594c
584.8 MB Download
md5:25ffac24562321ce6c90ea1333ac2899
556.8 MB Download
md5:6bad90d6bec73079d7892859d6c8cb8b
510.2 MB Download
md5:1eec14ba23f74a4af8e004ed9f4e0e1b
497.7 MB Download
md5:a2901f5006ebb0841338decaf8795861
296.7 MB Download
md5:b62fa55199f50a1db567ba7a40146530
434.4 MB Download
md5:bdf1904f71a323ac1abea3defde25f4a
444.9 MB Download
md5:fd60d0e308348929915fd95d5e1a6b3e
584.0 MB Download
md5:646786fea4a992948cce586e646b77a9
562.6 MB Download
md5:7700b1443d0fbd0bd95d4305adfcdbae
81.9 kB Download

Additional details

Funding

European Commission
REGALE - An open architecture to equip next generation HPC applications with exascale capabilities 956560
European Commission
EUPEX - EUROPEAN PILOT FOR EXASCALE 101033975
European Commission
The European PILOT - Pilot using Independent Local & Open Technologies 101034126
European Commission
DECICE - Device-Edge-Cloud Intelligent Collaboration framEwork 101092582
European Commission
Graph-Massivizer - Extreme and Sustainable Graph Processing for Urgent Societal Challenges in Europe 101093202