A preliminary empirical assessment of similarity for combinatorial interaction testing of software product lines

Autoren Stefan Fischer
Roberto E. Lopez-Herrejon
Rudolf Ramler
Alexander Egyed
Editoren
TitelA preliminary empirical assessment of similarity for combinatorial interaction testing of software product lines
BuchtitelProceedings of the 9th International Workshop on Search-Based Software Testing (SBST'16)
Typin Konferenzband
VerlagACM
ISBN978-1-4503-4166-0
DOI10.1145/2897010.2897011
MonatMay
Jahr2016
Seiten15-18
SCCH ID#1624
Abstract

Extensive work on Search-Based Software Testing for Software Product Lines has been published in the last few years. Salient among them is the use of similarity as a surrogate metric for t-wise coverage whenever higher strengths are needed or whenever the size of the test suites is infeasible because of technological or budget limitations. Though promising, this metric has not been assessed with real fault data. In this paper, we address this limitation by using Drupal, a widely used open source web content management system, as an industry-strength case study for which both variability information and fault data have been recently made available. Our preliminary assessment corroborates some of the previous findings but also raises issues on some assumptions and claims made. We hope our work encourages further empirical evaluations of Combinatorial Interaction Testing approaches for Software Product Lines.