On adequacy of assertions in automated test suites: An empirical investigation

Junji Zhi, Vahid Garousi

Research output: Contribution to conferencePaper

7 Citations (Scopus)

Abstract

An integral part of test case is the verification phase (also called 'test oracle'), which verifies program's state, output or behavior. In automated testing, the verification phase is often implemented using test assertions which are usually developed manually by testers. More precisely, assertions are used for checking the unit or the system's behavior (or output) which is reflected by the changes in the data fields of the class under test, or the output of the function under test. Originated from human (testers') error, test suites are prone to having inadequate assertions. The paper reports an empirical study on the Inadequate-Assertion (IA) problem in the context of automated test suites developed for open-source projects. In this study, test suites of three active open-source projects have been chosen. To investigate IA problem occurrence among the sampled test suites, we performed mutation analysis and coverage analysis. The results indicate that: (1) the IA problem is common among the sampled open-source projects, and the occurrence varies from project to project and from package to package, and (2) the occurrence rate of the IA problem is positively co-related with the complexity of test code.

Original languageEnglish
Pages382-391
DOIs
Publication statusPublished - 01 Aug 2013
Externally publishedYes
EventIEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013 - Luxembourg, Luxembourg
Duration: 18 Mar 201320 Mar 2013

Conference

ConferenceIEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013
CountryLuxembourg
CityLuxembourg
Period18/03/201320/03/2013

Keywords

  • adequacy of test oracles
  • automated testing
  • empirical case-study
  • mutation testing
  • quality of test oracles
  • state coverage
  • Test assertions
  • test-suite quality

ASJC Scopus subject areas

  • Software

Fingerprint Dive into the research topics of 'On adequacy of assertions in automated test suites: An empirical investigation'. Together they form a unique fingerprint.

  • Cite this

    Zhi, J., & Garousi, V. (2013). On adequacy of assertions in automated test suites: An empirical investigation. 382-391. Paper presented at IEEE 6th International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2013, Luxembourg, Luxembourg. https://doi.org/10.1109/ICSTW.2013.49