Design of a Semi-Automated System for Dietary Assessment of Image-Voice Food Records

Abstract

Smartphone-enabled image and voice recordings provide new avenues for capturing a person’s dietary intake. This can help to reduce the burdens of traditional dietary assessment methods (e.g., journals, observation), facilitating accessibility to under-represented populations. However, there is little research into how one can effectively process the image-voice data captured in this way for accurate dietary assessment. Building on an interdisciplinary team of experts in computing and nutrition, the goal of the present study is to co-design a web-based content management system to facilitate dietary assessment of image-voice dietary intake data. This led to the formulation and evaluation of five design principles for semi-automated dietary assessment of image-voice food records. Integrated into a wider project, the system was evaluated with actual dietary intake data collected in free-living conditions in Cambodia, and generative feedback provided by dietary assessment analysts. We discuss the challenges and experiences of building a system to process image-voice food records, and present the resulting design principles to aid future development.

Keywords

• Dietary assessment
• food recording
• natural language processing
• semi-automation
• speech-to-text
• system design