Crowdsourcing for self-monitoring: Using the Traffic Light Diet and crowdsourcing to provide dietary feedback

Abstract

Background Smartphone photography and crowdsourcing feedback could reduce participant burden for dietary self-monitoring. Objectives To assess if untrained individuals can accurately crowdsource diet quality ratings of food photos using the Traffic Light Diet (TLD) approach. Methods Participants were recruited via Amazon Mechanical Turk and read a one-page description on the TLD. The study examined the participant accuracy score (total number of correctly categorized foods as red, yellow, or green per person), the food accuracy score (accuracy by which each food was categorized), and if the accuracy of ratings increased when more users were included in the crowdsourcing. For each of a range of possible crowd sizes ( n = 15, n = 30, etc.), 10,000 bootstrap samples were drawn and a 95% confidence interval (CI) for accuracy constructed using the 2.5th and 97.5th percentiles. Results Participants ( n = 75; body mass index 28.0 ± 7.5; age 36 ± 11; 59% attempting weight loss) rated 10 foods as red, yellow, or green. Raters demonstrated high red/yellow/green accuracy (>75%) examining all foods. Mean accuracy score per participant was 77.6 ± 14.0%. Individual photos were rated accurately the majority of the time (range = 50%–100%). There was little variation in the 95% CI for each of the five different crowd sizes, indicating that large numbers of individuals may not be needed to accurately crowdsource foods. Conclusions Nutrition-novice users can be trained easily to rate foods using the TLD. Since feedback from crowdsourcing relies on the agreement of the majority, this method holds promise as a low-burden approach to providing diet-quality feedback.