Applied Sciences (May 2023)

Food Classification and Meal Intake Amount Estimation through Deep Learning

  • Ji-hwan Kim,
  • Dong-seok Lee,
  • Soon-kak Kwon

DOI
https://doi.org/10.3390/app13095742
Journal volume & issue
Vol. 13, no. 9
p. 5742

Abstract

Read online

This paper proposes a method to classify food types and to estimate meal intake amounts in pre- and post-meal images through a deep learning object detection network. The food types and the food regions are detected through Mask R-CNN. In order to make both pre- and post-meal images to a same capturing environment, the post-meal image is corrected through a homography transformation based on the meal plate regions in both images. The 3D shape of the food is determined as one of a spherical cap, a cone, and a cuboid depending on the food type. The meal intake amount is estimated as food volume differences between the pre-meal and post-meal images. As results of the simulation, the food classification accuracy and the food region detection accuracy are up to 97.57% and 93.6%, respectively.

Keywords