Multi-objective regression test selection in practice: An empirical study in the defense software industry

Vahid Garousi*, Ramazan Özkan, Aysu Betin-Can

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Citations (Scopus)
24 Downloads (Pure)

Abstract

Context: Executing an entire regression test-suite after every code change is often costly in large software projects. To cope with this challenge, researchers have proposed various regression test-selection techniques. Objective: This paper was motivated by a real industrial need to improve regression-testing practices in the context of a safety-critical industrial software in the defence domain in Turkey. To address our objective, we set up and conducted an “action-research” collaborative project between industry and academia. Method: After a careful literature review, we selected a conceptual multi-objective regression-test selection framework (called MORTO) and adopted it to our industrial context by developing a custom-built genetic algorithm (GA) based on that conceptual framework. GA is able to provide full coverage of the affected (changed) requirements while considering multiple cost and benefit factors of regression testing. e.g., minimizing the number of test cases, and maximizing cumulative number of detected faults by each test suite. Results: The empirical results of applying the approach on the Software Under Test (SUT) demonstrate that this approach yields a more efficient test suite (in terms of costs and benefits) compared to the old (manual) test-selection approach, used in the company, and another applicable approach chosen from the literature. With this new approach, regression selection process in the project under study is not ad-hoc anymore. Furthermore, we have been able to eliminate the subjectivity of regression testing and its dependency on expert opinions. Conclusion: Since the proposed approach has been beneficial in saving the costs of regression testing, it is currently in active use in the company. We believe that other practitioners can apply our approach in their regression-testing contexts too, when applicable. Furthermore, this paper contributes to the body of evidence in regression testing by offering a success story of successful implementation and application of multi-objective regression testing in practice.

Original languageEnglish
Pages (from-to)40-54
Number of pages15
JournalInformation and Software Technology
Volume103
Early online date30 Jun 2018
DOIs
Publication statusPublished - Nov 2018

Keywords

  • Action-research
  • Defence software industry
  • Empirical study
  • Genetic algorithms
  • Multi-objective optimization
  • Regression testing

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Multi-objective regression test selection in practice: An empirical study in the defense software industry'. Together they form a unique fingerprint.

Cite this