Image-based Approaches for Automating GUI Testing of Interactive Web-based Applications

Abstract

Modern Graphical User Interface testing frameworks automate the testing of web-based interfaces to ensure the absence of functional and visual regressions. They employ common strategies to tackle different testing use-cases. This paper presents an analysis of these strategies, highlighting that it is not possible to test some kinds of interactive interfaces exhaustively. We propose a novel image-based framework that combines current techniques with new ones. These leverage Machine Learning and Computer Vision algorithms to analyze screenshots of the interface and prove its correctness. Results suggest that it suffices to automate the verification of interactive interfaces that were not fully testable before. Automated tests, developed as a benchmark, present almost no false-positives and high accuracy.

Keywords

• computer vision
• ocr
• software testing
• visual testing
• user interface testing