Learning global image representation with generalized‐mean pooling and smoothed average precision for large‐scale CBIR

Abstract

Abstract Content‐based image retrieval (CBIR) is the problem of searching for items in an image database that are similar to the query image. Most of the existing image retrieval methods are trained based on metric learning loss functions (e.g. contrastive loss or triplet loss), however, which require the use of hard sample mining strategies (HMS) to better train the model. The HMS implies that picking out hard positive or negative samples increases the complexity of model training and requires a large amount of additional training time. To address this issue, lessons from recent work are leveraged on representation learning and a model called GS is proposed that combines the state‐of‐the‐art Generalized‐Mean (GeM) pooling and the smoothed average precision (AP). The entire network can be learned end‐to‐end by approximating the non‐differentiable AP function to a differentiable one‐without mining hard samples, only image‐level annotations. A model named GSA is also presented which achieves excellent retrieval performance jointly trained by two various loss functions. Experimental results validate the effectiveness of the proposed approach and demonstrate the competitive performance on a common standard image retrieval dataset (Revisited Oxford and Paris).

Keywords

• computer vision
• content‐based retrieval
• image retrieval