Published September 6, 2024 | Version v2
Dataset Open

The Effects of Computational Resources on Flaky Tests (Artifact)

Creators

Description

This artifact contains resources for reproducing and extending the work "The Effects of Computational Resources on Flaky Tests"

Contents:

  •  Analysis and Processed Test Results.tgz: An archive that contains information about the projects analyzed, summarized test results per-throttling configuration per-run, and a Jupyter notebook that detects RAFT (generating all tables and figures in the article). A README in this archive provides further guidance on its contents
  • js-results.tar, java-results.tar, python-results.tar: The raw results produced by the test runner when executing each JavaScript, Java and Python project 300 times in each of the throttling configurations.

See also:

We have published docker containers that include each project that we studied, along with all of the dependenices for running the tests. These containers can be used to reproduce our results, or to extend our work by running additional tests. The containers are available at https://hub.docker.com/r/jonbell/raft/tags

Files

Files (41.5 GB)

Name Size Download all
md5:c734a3104764c56be8d375024f2ddaf9
32.6 MB Download
md5:6931cc7acd5e53fcdf47b43d4163856e
26.9 GB Download
md5:a385988038e4a3ddcd410b92db14143d
3.7 GB Download
md5:27765e46267549410112b712bcb32bec
10.9 GB Download

Additional details

Funding

U.S. National Science Foundation
SHF: Medium: Collaborative Research: Enhancing Continuous Integration Testing for the Open-Source Ecosystem 2100037
U.S. National Science Foundation
CAREER: Amplifying Developer-Written Tests for Code Injection Vulnerability Detection 2100015
U.S. National Science Foundation
CAREER: Enhanced Reliability and Efficiency of Software Regression Testing in the Presence of Flaky Tests 2338287