IEEE Access (Jan 2024)
Refining a One-Parameter-at-a-Time Approach Using Harmony Search for Optimizing Test Suite Size in Combinatorial T-Way Testing
Abstract
In the pursuit of software quality, testing plays a pivotal role in identifying and rectifying errors and defects before software is delivered to users. However, the increasing complexity of modern software, driven by diverse user needs, has resulted in numerous input functions and options. While exhaustive testing remains ideal, practical limitations in terms of time and cost impede exhaustive efforts. To address these challenges, researchers have turned to metaheuristic algorithms to formulate effective t-way testing strategies, wherein ‘t’ denotes parameter interaction strength. These strategies encompass a range of exploitation and exploration techniques for generating test data. Despite these advancements, challenges persist in test case generation, such as enhancing software quality through parameter seeding and managing combinatorial explosion by understanding parameter constraints among system elements. Though existing metaheuristic-based t-way strategies offer valuable insights, none reign supreme over their counterparts. To bridge this gap, this article introduces a pioneering approach called SCHOP, integrating seeding and constraint supports within a harmony search algorithm by adopting a one-parameter-at-a-time approach. Benchmarking results illustrate the competitive performance of SCHOP across well-known benchmarking configurations against other strategies. SCHOP’s benchmarking results showed success in 61.07% of cases, with 80 out of 131 entries performing well. However, there were 51 instances of suboptimal test suite sizes (38.93%). Further analysis revealed statistical significance in 12 out of 61 cases, with 48 out of 61 cases showing no significant difference, all at a 95% confidence level ( $\alpha =0.05$ ). Finally, SCHOP concurrently integrates seeding and constraint support within the harmony search algorithm to boost software quality while reducing the test suite’s size.
Keywords
