Toward Improving the Quality of Mutation Operator and Test Case Effectiveness in Higher-Order Mutation Testing

Abstract

Currently, there are many research studies that apply and improve mutation testing techniques including traditional mutation testing or first-order mutation testing, and higher-order mutation testing (HOMT) for evaluating the quality of the set of test data in particular, and the quality of test suites in general. The results of those studies have proven the effectiveness of mutation testing in the field of software testing. Mutation testing allows the quality of test cases to be automatically evaluated, thereby helping the testers to improve the quality in the design and execution of the software testing. Besides, these studies have also pointed out the main barriers in applying mutation testing techniques in practice. However, we are the first to introduce a method that can reduce the cost, but keep the quality of testing activity based on evaluating the quality of the mutation operator as well as the quality of the test cases. In this paper, we concentrate on two problems regarding higher-order mutation testing: Evaluating the quality of mutation operators as well as generated mutants and prioritizing test cases based upon its capability of killing mutants. This may help developers allocate suitably their resources during testing phase. The study of this paper is an extended version of our previous study titled “Evaluating Mutation Operator and Test Case Effectiveness by Means of Mutation Testing”, which is published in the proceedings of the 13th Asian Conference on Intelligent Information and Database Systems, ACIIDS 2021 (V. N. Do, Q. V. Nguyen and T. B. Nguyen, Evaluating Mutation Operator and Test Case Effectiveness by Means of Mutation Testing, in Intelligent Information and Database Systems. ACIIDS 2021, eds. N. T. Nguyen, S. Chittayasothorn, D. Niyato and B. Trawiński. Lecture Notes in Computer Science, Vol. 12672 (Springer, Cham), https://doi.org/10.1007/978-3-030-73280-6_66) to confirm the usefulness of our proposed method.

Keywords

• Mutation testing
• second-order mutant
• mutation operator evaluation
• test case prioritization