CorrFractal: High-Resolution Correspondence Method Using Fractal Affinity on Self-Supervised Learning

Abstract

Existing supervised learning-based methods performed high-resolution visual correspondence using a decoder module. However, in self-supervised learning-based methods, it is difficult to use a decoder module that is easily influenced by labels. This paper will introduce a self-supervised learning-based visual correspondence method for high-resolution representation without decoder module. To this end, the paper proposed four modules. Each module has an output of the original resolution and distributes the role of the decoder module to perform high-resolution representation. The first module is the pattern boosted quantization module, which learns pattern information along with color information to create high-resolution pseudo labeling. The second module is the backbone module, which is created by applying aggregation to the backbone network to simultaneously handle semantic features and high-resolution features. The third module is the appearance module, which learns appearance information using the features of the high-resolution embedding space. The fourth module is the correspondence module, which gradually reconstructs a high-resolution visual correspondence using low-resolution input. It was confirmed using subtraction image that the proposed method improves the performance about representation of thin objects and object boundaries. Video segmentation performance was evaluated on the DAVIS-2017 val dataset using the J&F mean, yielding 65.4%.

Keywords

• Decoder module
• high-resolution representation
• pseudo labeling
• self-supervised learning
• visual correspondence