Focusing on Valid Search Space in Open-World Compositional Zero-Shot Learning by Leveraging Misleading Answers

Abstract

The goal of Compositional Zero-Shot Learning (CZSL) is to recognize various compositions of state-object pairs. Because the compositions that need to be considered are only a subset of all combinations of states and objects, it is tough for models to predict unseen compositions. Previous work overlooks the problem of predicting non-sensical compositions such as flying dogs. To address this problem, we introduce a novel method for the model to distinguish between target and non-target composition space to avoid predicting absurd compositions. More specifically, in the process of predicting the states and objects, we train the model to increase the similarity with the label that matches the input image while decreasing the similarity with non-matched labels. Our method calculates the logits for the composition labels by combining the similarities of the image-states and the similarities of image-objects respectively. Then, the combined logits and directly computed composition logits are used to minimize the case of the predicting absurd composition. On three well-known datasets such as MIT-States, UT-Zappos, and C-GQA, various experimental results demonstrate our simple and novel approach significantly improves model performances. Code is available at: https://github.com/ToBeSuperior/Annotation-embedding

Keywords

• Compositional zero-shot learning
• open-world recognition
• representation learning
• vision and language
• zero-shot learning