USRRM: Pairwise Ranking and Scoring Images Using Its Aesthetic Quality

Abstract

Image Aesthetics Analysis is a challenging research problem as aesthetics of an image is a subjective quality and it is quite difficult to formulate it into a mathematical or algorithmic problem. On the other hand, its applications are numerous, ranging from aesthetic based image retrieval to image editing. Earlier works on image aesthetics analysis relied upon handpicking the standard features from the image, based upon which, its aesthetics was quantified. This method was applied, in the belief that sufficient aesthetic features have been taken into consideration and no more features impact its aesthetic quality. This is not always the case, since, subjective quality as this, is defined individually and depends upon personal perspective. With the advent of deep learning, automatic feature learning became prevalent. The classification works on image aesthetics using deep learning have had some success. However, we are concerned with giving a tentative score to images and perform some sort of relative ranking among them. We formulate a unified loss objective accounting both of these factors and also devise a model named USRRM which learns from the global view and pixel-level finer details from an image. The external style and semantic information also aid in model learning. The scoring result of USRRM has the best correlation with the ground truth scores among the most similar previous models and comparable results with less similar models but evaluated with the same metrics. We quantify our results using different correlation coefficients and an accuracy metric.

Keywords

• Convolutional neural network
• image aesthetics
• image ranking