A search-based approach for cost-effective software test automation decision support and an industrial case study

Yasaman Amannejad, Vahid Garousi, Rob Irving, Zahra Sahaf

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Citations (Scopus)
54 Downloads (Pure)

Abstract

Test automation is a widely-used approach to reduce the cost of manual software testing. However, if it is not planned or conducted properly, automated testing would not necessarily be more cost effective than manual testing. Deciding what parts of a given System Under Test (SUT) should be tested in an automated fashion and what parts should remain manual is a frequently-asked and challenging question for practitioner testers. In this study, we propose a search-based approach for deciding what parts of a given SUT should be tested automatically to gain the highest Return On Investment (ROI). This work is the first systematic approach for this problem, and significance of our approach is that it considers automation in the entire testing process (i.e., from test-case design, to test scripting, to test execution, and test-result evaluation). The proposed approach has been applied in an industrial setting in the context of a software product used in the oil and gas industry in Canada. Among the results of the case study is that, when planned and conducted properly using our decision-support approach, test automation provides the highest ROI. In this study, we show that if automation decision is taken effectively, test-case design, test execution, and test evaluation can result in about 307%, 675%, and 41% ROI in 10 rounds of using automated test suites.

Original languageEnglish
Title of host publicationProceedings - IEEE 7th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2014
PublisherIEEE Computer Society
Pages302-311
Number of pages10
ISBN (Print)9780769551944
DOIs
Publication statusPublished - 01 Jan 2014
Externally publishedYes
Event7th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2014 - Cleveland, OH, United States
Duration: 31 Mar 201404 Apr 2014

Conference

Conference7th IEEE International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2014
Country/TerritoryUnited States
CityCleveland, OH
Period31/03/201404/04/2014

Keywords

  • action research
  • cost-benefit analysis
  • industrial case study
  • search-based software engineering
  • software test automation

ASJC Scopus subject areas

  • Software

Fingerprint

Dive into the research topics of 'A search-based approach for cost-effective software test automation decision support and an industrial case study'. Together they form a unique fingerprint.

Cite this