Improvement in automatic food region extraction based on saliency detection

Abstract

In this paper, we propose a method for extracting pixel-wise food regions on the basis of saliency detection using the multi-scale information network (MSI-Net), which includes convolutional layers at different dilated rates, and GrabCut, which can revise food regions on the basis of graph theory. Our comparative experiment, which used 241 actual food images, clarified that the proposed method significantly increased the F-measure, which was used as a comprehensive metric of the food-extraction accuracy, by 3.76% or more compared with conventional methods using saliency detection. The proposed method tended to preserve the contour structure of the food regions better than the conventional methods. In addition, the F-measure was significantly increased by 6.72% compared with the low-cost SegNet, which was trained with a publically available dataset. However, the experiment suggested that the proposed method can be further improved by revising its algorithm for determining food and background candidates. Further discussion and implications are provided herein.

Keywords

• Ood image
• Food region extraction
• segmentation
• GrabCut
• Food record