Replication package

Applicability of Search-based Algorithms for Model-based Game Play Testing - an Empirical Study

Folders

• Data: the folder contains raw experimental data
• Rscripts: the folder contains all the R scripts used to analyze data the and to generate the plots
• OutTables: the folder is created when running scripts in Rscripts folder and contains all the generated tables
• Plots: the folder is created when running scripts in Rscripts folder and contains all the generated plots
• TexFolder folder contains tables in tex format
• Cytoscape folder contains graphs used in Figure 11

Data folder

The folder Data contains raw experimental data and includes subfolders Experiments and runOnSUT.

Run on model

Experiments folder contains raw data execution on EFSM models. In particular, folder 20220304 contains 4 tables

• efsm_features: for each configuration reports the values of the EFSM features
• experiments_parameters: set EFSM random generator parameters
• experiments_third_large_20220304_silent: list of the EvoMBT parameters for each run
• run_statistics: execution statistics of EvoMBT

Run on SUT

Folder runOnSUT is compressed in file runOnSUT.zip to save space. One uncompressed, folder runOnSUT occupies ~1GB.

Running R scripts

Requirements:

• R scripts are tested with R version 4.2.2
• the scripts require the following packages are installed: data.table, ggplot2, snow, parallel, effsize, RColorBrewer, caret, reshape2, scales, rstatix, PMCMRplus, ggpubr, cowplot, bit64, circlize, ComplexHeatmap, GGally, cluster, factoextra, NbClust, xtable

To run R scripts have to be run from folder Rscripts. Scripts need to be executed in a specific order as a script can prepare the data for another script. Finally, depending on operating system, R script can be invoked using Rscript or Rscript.exe. In the following, we will use Rscript to indicate both.

Decompress Experiments.zip in folder Data. Preprocess raw experimental data on EFSM models.

Rscript preprocess_data.R

RQ1 data analysis.

Rscript rq1_data_analysis.R

RQ1.1 data analysis.

Rscript rq1_1_data_analysis.R

RQ2 data analysis.

Rscript rq2_data_analysis.R

Decompress runOnSUT.zip in folder Data. Preprocess experimental data run on LabRecruits.

Rscript preprocess_run_on_sut_data.R

RQ3 data analysis.

Rscript rq3_data_analysis.R