Shaken, Not Stirred. How Developers Like Their Amplified Tests: Replication Package

# Replication Package for "Shaken, Not Stirred: How Developers Like Their Amplified Tests"

Welcome to our replication package! 😊

This readme will give you an overview on what to find where in this package, especially the points we mention in the paper.
For each step of the study, we provide dedicated readmes with more details.

## Where to find what we referenced in the paper

### Section III.B: Full Templates for Descriptions and our Implementations

Our implementation is open source on GitHub (link will be provided after double-blind review).

We include the DSpot and prettifier versions used for the study under [`tools/dspot`](tools/dspot) (compiled .jars and the full code), and a full overview of the description template in [`tools/description-template-full.md`](tools/description-template-full.md).

### Section IV.B: Exact Configuration of DSpot and Prettifier + Hardware used in Study

The exact configuration of DSpot and the Prettifier are set in [`tools/sh-dspot.sh`](tools/sh-dspot.sh) and [`tools/sh-prettifier.sh`](tools/sh-prettifier.sh), respectively.

For our study, we ran the test amplification and prettification on a desktop PC (Windows 10, WSL2, Intel i9-11900K, 16GB RAM), a time budget of 30 minutes and the software versions mentioned in the requirements above. (also noted in [`readme-test-generation.md`](readme-test-generation.md)).

### Section IV.E: Codebook---Outcome of our Pull Request Analysis

We provide the full lists of the codes related to RQ2.1, RQ2.2 and RQ2.3 (from the pull request discussions) in [`CodeBook.xlsx`](CodeBook.xlsx).
For an easy look at how frequently each high-level code occurred, refer to [`readme-data-analysis.md`](CodeBook.xlsx) (## PR Analysis).

Currently, we cannot make the full screenshots available as they would unblind the authors who opened the pull requests under their real name.
We will make the whole set of screenshots, connected to the codes, available after the acceptance of the paper.

You find an overview of the final [selection](manual_editing_phase/SelectionChecklist.xlsx) and [editing](manual_editing_phase/EditingChecklist.xlsx) checklists in the folder `manual_editing_phase`.
And again an easy frequency overview in the same data analysis :).

## Overview of the Whole Package Structure

`csv`: input files for the various steps throughout the test amplification process, produced by the scripts provided in this package.

`data_analysis`: python scripts for the data analysis, producing the tables and results visible in the paper

`generating_tests`: python scripts to facilitate the test amplification + prettification for our study

`gh_quality_analytics`: (relevant parts of) the dataset we use for the initial selection of OS Java repositories

`manual_analysis_validation`: the files from our manual analysis and cross-validation between the authors after the first full run of DSpot

`manual_editing_phase`: all the data from our manual preparation phase for the pull requests & the pull request discussions (amplified tests, selected and edited tests, submitted patches, pull requests and observations)

`study_data`: input, intermediate and output data from our study that we save separately to keep the analysis scripts working on our data when re-running the test amplification

`tools`: the test amplification tools we use (compiled + full code), as well as helper scripts to configure them and set up the small demo