TESRAC: A Framework for Test Suite Reduction Assessment at Scale

João Becho, Frederico Cerveira, João Leitão, Rui André Oliveira

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

Regression testing is an important task in any large software project, however as codebase increases, test suites grow and become composed of highly redundant test cases, thus greatly increasing the time required for testing. To solve this problem various test suite reduction tools have been proposed, however their absolute and relative performance are unclear to their prospective users, since there is a lack of a standardized evaluation or approach for choosing the best reduction tool. This work proposes TESRAC, a framework for assessing and comparing test suite reduction tools, which allows users to evaluate and rank a customizable set of tools in terms of reduction performance according to criteria (coverage, dimension, and execution time), and which can be configured to prioritize specific criteria. We used TESRAC to assess and compare three test suite reduction tools and one test suite prioritization tool that has been adapted to perform test suite reduction, across eleven projects of various dimensions and characteristics. Results show that a test suite prioritization tool can be adapted to perform a adequate test suite reduction, and a subset of tools outperforms the remaining tools for the majority of the projects. However, the project and test suite being reduced can have a strong impact on a tool's performance.

Original languageEnglish
Title of host publication2022 IEEE 15th International Conference on Software Testing, Verification and Validation (ICST) Proceedings
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages174-184
Number of pages11
ISBN (Electronic)978-1-6654-6679-0
ISBN (Print)978-1-6654-6680-6
DOIs
Publication statusPublished - Apr 2022
Event15th IEEE International Conference on Software Testing, Verification and Validation - Valência, Spain
Duration: 4 Apr 202214 Apr 2022

Conference

Conference15th IEEE International Conference on Software Testing, Verification and Validation
Abbreviated titleICST 2022
Country/TerritorySpain
CityValência
Period4/04/2214/04/22

Keywords

  • evaluation
  • software testing
  • test case prioritization
  • test suite minimization
  • test suite reduction

Fingerprint

Dive into the research topics of 'TESRAC: A Framework for Test Suite Reduction Assessment at Scale'. Together they form a unique fingerprint.

Cite this