Published May 25, 2026 | Version v2

Exploring Statistical Change Point Detection Techniques for Performance Anomaly Detection at Mozilla

  • 1. ROR icon Concordia University

Description

Dataset description
This repository contains datasets collected as part of a study investigating performance regression detection and change point evaluation in software performance monitoring systems. The data originates from controlled annotation tasks and follow-up surveys conducted with Mozilla engineers.

The dataset is composed of 2 main components:

  • Annotation Experiment Data (tasks_anonymized.csv and annotations_anonymized.csv)
  • Timeseries data & attributes (timeseries.zip and timeseries_attributes.json)

Annotation Experiment Data
This dataset captures the results of a manual annotation experiment where participants analyzed performance datasets and identified potential change points. It consists of two CSV files:

  • Tasks Dataset (tasks_anonymized.csv): The tasks dataset records the annotation tasks assigned to participants and metadata related to their completion. Its columns are:
Column Description
TaskID Unique identifier of the annotation task
DatasetName Identifier of the performance dataset analyzed in the task
Difficulty Self-reported difficulty level of the task
TimeSpent Time spent completing the task (in seconds)
Problem Optional field for reporting issues encountered during the task
UserID Identifier of the participant who completed the task



  • Annotations Dataset (annotations_anonymized.csv): The annotations dataset contains the actual change point annotations produced by participants while analyzing the datasets. This dataset is used to construct the ground truth annotations and analyze agreement between annotators. Its columns are:
Column Description
DatasetName Identifier of the dataset being annotated
UserID Identifier of the participant who made the annotation
AnnotationIndex Index position of the annotated change point within the time series
AnnotationType Type of change detected (mean, variance, and mean_variance)


It is worth noting that the variance annotations were dropped upon doing the evaluation of the change point detection methods mentioned in the paper as there was low agreement on this type of annotation points.

  • Timeseries data & attributes (timeseries.zip and timeseries_attributes.json): They corespond to the time series data used in the annotations process (a total of 174 time series), and their attreibutes which some of them were displayed to annotators upon performing the annotation tasks. Refer to this publication and its replication package here for more context on the time series data.

Files

dataset.zip

Files (3.4 MB)

Name Size
md5:7b228bbf49422aefa4bffae3548a1c5e
3.4 MB Preview Download

Additional details

Related works

Cites
Data paper: 10.1145/3680256.3721973 (DOI)
Is supplement to
Dataset: 10.5281/zenodo.15465568 (DOI)

Dates

Available
2026-05-25

Software