Hybrid Attention-Based Prototypical Network for Unfamiliar Restaurant Food Image Few-Shot Recognition

Abstract

As eating-out became an indispensable part of our daily lives, demand for the food recognition of unfamiliar restaurant increased significantly due to health-care. Although there are many researches on generic food recognition, there are relatively fewer studies on restaurant food image recognition. Meanwhile, it becomes extremely challenging for restaurant food image recognition due to insufficient food image. Prototypical network is common utilized to address the such a task in recent years. Although the methods based on prototypical network achieve impressive results in capturing similarities feature of the same food category, it fails to highlight important information on feature and instance level. Toward this end, we propose an effective food image recognition scheme by incorporating hybrid attention mechanism into prototypical network in this paper. Specifically, the image feature is first captured by convolutional neural network (CNN). Then the image attention weights yielded by instance-based attention mechanism are used to modulate the image feature of CNN for constructing class prototypes. And feature-based attention mechanism is employed to grasp important information of image for enriching image representation. Extensive experimental results on the large food image dataset verify that the performance of our proposed classification scheme outperforms the state-of-the-art ones.

Keywords

• Restaurant food image recognition
• unfamiliar restaurant
• prototypical network
• hybrid attention mechanism