Transitioning from Manual to Automated Software Regression Testing: Experience from the Banking Domain

Regression testing is needed when a software or the environment hosting that software changes. Motivated by a real-world industrial need in the context of a large financial (banking) corporation in Turkey, the authors and their colleagues developed and introduced an automated regression testing infrastructure for automated testing of one of the main mobile applications of the company. Before this project, regression testing was conducted manually which incurred a lot of costs and was by nature subjective. We report in this paper our experience in 'transitioning' from manual to automated regression testing, and in developing and introducing a set of large automated test suites (more than 16 KLOC in total), using best practices in state-of-the art and -practice, and to report its observed benefits by conducting cost-benefit analysis. The project was conducted based on the principles of case-study and 'action research' in which the real industrial needs drove the research. Among the best practices that we used are the followings: (1) modularity in test code, (2) creating test-specific libraries, and (3) separating test data from test logic. By serving as a success story and experience report in development and introduction of automated test suites in an industrial setting, this paper adds to the body of evidence in this area and it aims at sharing both technical (e.g., using automated test patterns) and process aspects (e.g., test process improvement) from our project with other practitioners and researchers.