Exploring Statistical Change Point Detection Techniques for Performance Anomaly Detection at Mozilla
Authors/Creators
Description
Dataset description
This repository contains datasets collected as part of a study investigating performance regression detection and change point evaluation in software performance monitoring systems. The data originates from controlled annotation tasks and follow-up surveys conducted with Mozilla engineers.
The dataset is composed of 2 main components:
- Annotation Experiment Data (tasks_anonymized.csv and annotations_anonymized.csv)
- Timeseries data & attributes (timeseries.zip and timeseries_attributes.json)
Annotation Experiment Data
This dataset captures the results of a manual annotation experiment where participants analyzed performance datasets and identified potential change points. It consists of two CSV files:
- Tasks Dataset (tasks_anonymized.csv): The tasks dataset records the annotation tasks assigned to participants and metadata related to their completion. Its columns are:
| Column | Description |
TaskID |
Unique identifier of the annotation task |
DatasetName |
Identifier of the performance dataset analyzed in the task |
Difficulty |
Self-reported difficulty level of the task |
TimeSpent |
Time spent completing the task (in seconds) |
Problem |
Optional field for reporting issues encountered during the task |
UserID |
Identifier of the participant who completed the task |
- Annotations Dataset (annotations_anonymized.csv): The annotations dataset contains the actual change point annotations produced by participants while analyzing the datasets. This dataset is used to construct the ground truth annotations and analyze agreement between annotators. Its columns are:
| Column | Description |
DatasetName |
Identifier of the dataset being annotated |
UserID |
Identifier of the participant who made the annotation |
AnnotationIndex |
Index position of the annotated change point within the time series |
AnnotationType |
Type of change detected (mean, variance, and
mean_variance) |
It is worth noting that the variance annotations were dropped upon doing the evaluation of the change point detection methods mentioned in the paper as there was low agreement on this type of annotation points.
- Timeseries data & attributes (timeseries.zip and timeseries_attributes.json): They corespond to the time series data used in the annotations process (a total of 174 time series), and their attreibutes which some of them were displayed to annotators upon performing the annotation tasks. Refer to this publication and its replication package here for more context on the time series data.
Files
dataset.zip
Additional details
Related works
- Cites
- Data paper: 10.1145/3680256.3721973 (DOI)
- Is supplement to
- Dataset: 10.5281/zenodo.15465568 (DOI)
Dates
- Available
-
2026-05-25
Software
- Repository URL
- https://github.com/mozilla/REALISE-Performance
- Development Status
- Active