Study of Visual Quality Assessment on Pattern Images: Subjective Evaluation and Visual Saliency Effects

Abstract

This paper presents a comparison study on visual quality scores obtained from single-stimulus and double-stimulus approaches for pattern images, respectively. We also conduct the comparison on general (non-pattern) images, which serve as a control group. The pattern and non-pattern (PNP) images are collected and built by the Perceptual Data Analysis and Processing Laboratory, National Chung Hsing University, and evaluated by a group of people with both single-stimulus and double-stimulus approaches. Then, we examine the difference of mean opinion scores obtained by both the methods with respect to the image types, image contents (scenes), and distortion types. The hypothesis tests are used to determine whether there is a significant difference between both the sets of scores. The test results suggest that the differences exist and are significant for some specific distortion types and image contents. For example, the pattern images with simple colors should be viewed by the double-stimulus method in order to reduce the variance of subjective scores. In the end, we test three well-known full-reference image quality metrics (IQMs) on both the PNP images. In addition, we investigate whether the visual saliency (VS) plays a more important role for pattern images compared with non-pattern images. We discover that the most salient regions are more influential on pattern images than non-pattern images. Also, by introducing the VS information into the IQMs with a newly proposed Otsu's weighted VS mask, the correlation performance between objective quality scores obtained from IQMs and subjective scores from human's visual perception can be boosted further.

Keywords

• Absolute category rating-hidden reference (ACR-HR)
• hypothesis test
• pattern images
• simultaneous double stimulus for continuous evaluation-discrete category scale (SDSCE-DCS)
• visual saliency (VS)