Published October 11, 2025 | Version v2

Choosing the Right Test Generation Tool: A Guide for Software Practitioners

  • 1. ROR icon Universidade de Brasília
  • 2. ROR icon Aalto University
  • 3. ROR icon Instituto Federal de Educação Ciência e Tecnologia do Paraná

Description

 Context: As software systems become increasingly complex, testing automation plays a critical role in ensuring product quality and reliability. However, the wide variety of available test generation tools—with distinct purposes, features, and approaches—makes it difficult for practitioners to select the most appropriate option for their projects. Goal: This study aims to identify, analyze, and compare test generation tools reported in both academic and industrial contexts, providing a practical reference guide to support software professionals in tool selection decisions.  Method: We conducted a Multivocal Literature Review (MLR) complemented by a two-round survey with 87 software practitioners. The MLR identified tools and features reported in white and gray literature, while the survey assessed practitioners’ familiarity, usage, and perceptions of advantages and challenges associated with these tools. Results: The findings reveal a persistent gap between academic proposals and industrial adoption. While tools such as Postman, Selenium, and Cypress are among the most widely used tools in practice, academic tools like Monkey and Dynodroid remain rarely adopted. Practitioners value visibility, integration, and traceability as key features, whereas inconsistency and maintenance effort are seen as primary challenges. Conclusions: The study contributes a structured reference guide to assist professionals in selecting tools suitable for specific testing contexts. It also provides insights for researchers aiming to align future tool development with industry needs, fostering better usability, integration, and sustainability of automated testing solutions.

Files

MLR_Protocol.pdf

Files (1.0 MB)

Name Size Download all
md5:b4f312c1690f4d2d34bca528fb803ab9
455.9 kB Preview Download
md5:d40dabe09c9a086ad5e17c16913b029f
274.7 kB Preview Download
md5:1be9a350469d7d8690ff7e4b4b4e618f
216.8 kB Preview Download
md5:abf093e35006dff0eccedea020b97dda
17.9 kB Download
md5:a5647b5670b40080d2001a405cfd0b5c
44.8 kB Download

Additional details

Dates

Available
2024-06-18