Strong mutants and Subsuming mutants study for EPL language

Mutation testing has been applied successfully to several programming languages. Despite its benefits for software testing, its high computational cost has prevented it from being widely used. Several refinements have been proposed to reduce its cost by decreasing the number of generated mutants, one of which is Evolutionary Mutation Testing. This refinement aims to generate a reduced set of mutants with an Evolutionary Algorithm, which searches for potentially equivalent and difficult to kill mutants, defined as strong mutants, that help improve the test suite. This study presents Tempus, a system that includes a Profile-Based individual generation version of Evolutionary Mutation Testing by Domínguez-Jiménez et al. (2011). Tempus has been applied to four case studies that process information in real time from Internet of Things systems. This huge volume of information arrives as events that need to be monitored and processed in real time: the case studies manage the events through Esper Event Processing Language queries. Given that the events used as input in the Internet of Things systems may need a reaction in a specific period of time, it is crucial to test that the system can trigger the expected responses on demand within the expected period.

In this dataset the following is included (Tempus folder):

• The four case studies.

• The configuration of the four case studies for the application of Tempus.

• The input events used for each case study.

• The output of each case study after applying Tempus.

•  

  The Tempus tool, composed of MuEPL and GAmera.

In terms of judging the value of a mutant (in terms of motivating the design of test cases), subsuming mutants have played an important role in the literature. Ammann (2014) defined m_x dynamically subsumes m_y (i.e. m_x subsumes m_y if mutant x was not live and every test case that killed x also killed y. Papadakis (2016) raised concerns that using the subsumed mutants for mutation-based test assessment could artificially inflate the ability of a test technique to detect faults.

In summary: strong mutants are for guiding a search-based algorithm to identify mutants that could be killed, whereas subsuming mutants are for comparing the thoroughness of different test suites. The comparisons in this work have been based on the time spent by various configurations of EMT to find strong mutants, rather than on the mutant-killing power of any test suite. Incidentally, we have provided a first evaluation of the MuEPL mutation operators in terms of their production of subsuming mutants.

In this data set the following is included (Subsuming folder):

• A Python guide to analyze the subsuming mutants.

• The intermediate results obtained during the subsuming mutant analysis process.

• The raw data corresponding to the four case studies.

Domínguez-Jiménez JJ, Estero-Botaro A, García-Domínguez A, Medina-Bulo I (2011) Evolutionary mutation testing. Information and Software Technology 53(10)

Ammann P, Delamaro ME, Offutt J (2014) Establishing Theoretical Minimal Sets of Mutants. In: Verification and Validation 2014 IEEE Seventh International Conference on Software Testing, pp 21–30, , iSSN: 2159-4848

Papadakis M, Henard C, Harman M, Jia Y, Le Traon Y (2016) Threats to the validity of mutation-based test assessment. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, Association for Computing Machinery, New York, NY, USA, ISSTA 2016, pp 354–365