A genetic algorithm-based stress test requirements generator tool and its empirical evaluation

Vahid Garousi*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

21 Citations (Scopus)

Abstract

Genetic algorithms (GAs) have been applied previously to UML-driven stress test requirements generation with the aim of increasing chances of discovering faults relating to network traffic in distributed real-time systems. However, since evolutionary algorithms are heuristic, their performance can vary across multiple executions, which may affect robustness and scalability. To address this, we present the design and technical detail of a UML-driven, GA-based stress test requirements generation tool, together with its empirical analysis. The main goal is to analyze and improve the applicability, efficiency, and effectiveness and also to validate the design choices of the GA used in the tool. Findings of the empirical evaluation reveal that the tool is robust and reasonably scalable when it is executed on large-scale experimental design models. The study also reveals the main bottlenecks and limitations of the tools, e.g., there is a performance bottleneck when the system under test has a large number of sequence diagrams which could be triggered independently from each other. In addition, issues specific to stress testing, e.g., the impact of variations in task arrival pattern types, reveal that the tool generally generates effective test requirements, although the features of those test requirements might be different in different runs (e.g., different stress times from the test start time might be chosen). While the use of evolutionary algorithms to generate software test cases has been widely reported, the extent, depth, and detail of the empirical findings presented in this paper are novel and suggest that the proposed approach is effective and efficient in generating stress test requirements. It is hoped that the findings of this empirical study will help other SBSE researchers with the empirical evaluation of their own techniques and tools.

Original languageEnglish
Article number5383373
Pages (from-to)778-797
Number of pages20
JournalIEEE Transactions on Software Engineering
Volume36
Issue number6
DOIs
Publication statusPublished - 05 Jul 2010
Externally publishedYes

Keywords

  • empirical analysis
  • genetic algorithms
  • Index TermSearch-based testing
  • stress testing
  • test automation
  • test tools

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'A genetic algorithm-based stress test requirements generator tool and its empirical evaluation'. Together they form a unique fingerprint.

Cite this