Testing Asynchronous Reactive Systems: Beyond the ioco framework

Abstract

Manual testing can be rather time consuming and prone to errors specially when testing asynchronous reactive systems. Model based testing is a well-established approach to verify reactive systems specified by input output labeled transition systems (IOLTSs). One of the challenges stemming from model based testing is verifying conformance and, also, generating test suites, primarily when completeness is a required property. In order to check whether an implementation under test is in compliance with its respective specification one resorts to some form of conformance relation that guarantees the expected behavior of the implementations, given the behavior of the specification. The ioco relation is an example of such a conformance relation. In this work we study another conformance relation based on formal languages. We also investigate how to generate finite and complete test suites for IOLTS models, and discuss the complexity of the test generation mechanism under this new conformance relation. We also show that ioco is a special case of this new conformance relation. Further, we relate our contributions to more recent works, accommodating the restrictions of their classes of fault models as special cases, and we expose the complexity of generating any complete test suite that must satisfy their restrictions.

Keywords

• Test suite generation
• Complete test suites
• Asynchronous systems
• IOLTS