TUD Telemetry Dataset for Anomaly Detection

Dataset description

This dataset contains snapshots of host telemetry metrics collected during different workload conditions. It is intended for training and evaluating anomaly detection models (e.g., reconstruction-based autoencoders).

The metrics cover:

• Per-core CPU utilization breakdown by state (percent)
• Memory metrics (bytes)

Files

This Zenodo record should include the dataset in CSV format and this README.md.

Recommended file naming (adjust to your actual file names):

• train_no_load.csv, train_medium_load.csv, train_high_load.csv: training CSVs for different workload scenarios
• test_telemetry.csv: test CSV used for evaluation

If you prefer a single file, you can also concatenate all rows into one telemetry_dataset.csv and add two extra columns:

• scenario in {no_load, medium_load, high_load, ...}
• split in {train, test}

How the dataset was generated

1. A node was instrumented with a telemetry pipeline (e.g., Prometheus + node exporter) to collect CPU and memory metrics at a fixed sampling interval.
2. Multiple workload scenarios were executed (e.g., no load / medium load / high load).
3. Metrics were exported to CSV with a fixed column order. Each row represents one telemetry snapshot.

Notes:

• CPU values are percentages per core and CPU state. Due to sampling/aggregation, values may occasionally slightly exceed 100.
• node_memory_MemTotal_bytes is constant for a given machine (total installed memory).

Columns

All CSV files share the same schema (38 columns). Units and meanings are listed below.

CPU columns (percent)

For each core i in {0,1,2,3}, the following columns represent the percentage of time spent in the given CPU state during the sampling window:

• cpu_i_idle, cpu_i_iowait, cpu_i_irq, cpu_i_nice, cpu_i_softirq, cpu_i_steal, cpu_i_system, cpu_i_user

Memory columns (bytes)

• memory_used_bytes: used memory in bytes (as exported by the telemetry pipeline)
• node_memory_Buffers_bytes: memory used for buffers
• node_memory_Cached_bytes: memory used for page cache
• node_memory_MemAvailable_bytes: estimate of memory available for starting new applications
• node_memory_MemFree_bytes: unused memory
• node_memory_MemTotal_bytes: total installed memory

Column descriptions (full list)

Summary statistics

The following table reports per-column data type and summary statistics (min / median / max).

This table was computed from the provided file.

Reproducing the statistics table

To recompute the summary statistics for one or more CSV files (e.g., all training files plus the test file), run the following locally (requires pandas and numpy):

python - <<"PY"
import glob
import numpy as np
import pandas as pd

# Edit paths as needed
files = glob.glob("data/*.csv") + ["telemetry.csv"]

frames = [pd.read_csv(f) for f in files]
df = pd.concat(frames, ignore_index=True, sort=False)

rows = []
for col in df.columns:
    s = df[col]
    dtype = str(s.dtype)
    if pd.api.types.is_numeric_dtype(s):
        arr = s.to_numpy(dtype=float)
        rows.append((col, dtype, np.nanmin(arr), np.nanmedian(arr), np.nanmax(arr)))
    else:
        rows.append((col, dtype, np.nan, np.nan, np.nan))

print("| Column | Type | Min | Median | Max |")
print("|---|---:|---:|---:|---:|")
for col, dtype, mn, med, mx in rows:
    def fmt(v):
        if isinstance(v, float) and np.isnan(v):
            return ""
        av = abs(float(v))
        if av >= 1e6:
            return f"{v:.6g}"
        return f"{v:.4g}"

    print(f"| `{col}` | `{dtype}` | {fmt(mn)} | {fmt(med)} | {fmt(mx)} |")
PY