Platemate: crowdsourcing nutritional analysis from food photographs

By John Noronha, Eric Hysen, Haoqi Zhang, Krzysztof Z Gajos. ACM Symposium on User Interface Software and Technology, 2011

295 citations recorded by the source corpus.

We introduce PlateMate, a system that allows users to take photos of their meals and receive estimates of food intake and composition. Accurate awareness of this information can help people monitor their progress towards dieting goals, but current methods for food logging via self-reporting, expert observation, or algorithmic analysis are time-consuming, expensive, or inaccurate. PlateMate crowdsources nutritional analysis from photographs using Amazon Mechanical Turk, automatically coordinating untrained workers to estimate a meal's calories, fat, carbohydrates, and protein. We present the Management framework for crowdsourcing complex tasks, which supports PlateMate's nutrition analysis workflow. Results of our evaluations show that PlateMate is nearly as accurate as a trained dietitian and easier to use for most users than traditional self-reporting.

Subfields: Computer Science Applications, Transportation, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/632ffa1766498ba20a5e00138a891ba11520d35c

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Science Applications

1. Adam Marcus
2. Aditya G. Parameswaran
3. Li Fei-Fei
4. Olga Russakovsky
5. Michael S. Bernstein
6. Alia Al-Khatib
7. Margaret Levi
8. Melissa A. Valentine

In Transportation

1. Michael S. Bernstein
2. Olga Russakovsky
3. Li Fei-Fei

In Human-Computer Interaction

1. Li Fei-Fei
2. Michael S. Bernstein
3. Aditya G. Parameswaran
4. Adam Marcus
5. Jia Deng
6. Luc Van Gool
7. Sergio Guadarrama
8. Kevin P. Murphy

Citations in the corpus

• Food-101 - Mining Discriminative Components with Random Forests (2014, background): “However, current systems resort to nutrition experts [27] or Amazon Mechanical Turk [30] to label food items.”
• Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms (2015, background): “How does the platform enable tasks of increasing complexity to be effectively completed by crowds (Noronha et al., 2011; Heer & Bostock, 2010)?”
• Hierarchical Subquery Evaluation for Active Learning on a Graph (2014, background): “e is how to procure enough training labels to tune a semi-supervised model for a specified classification task. While unskilled Mechanical Turkers are willing to label images of food at $1.40 per image [25], the costs are massive for recruiting and paying specialists like doctors or scientists. Whethe”
• BodyScope: a wearable acoustic sensor for activity recognition (2012, background): “developed Platemate, which allows the user to analyze her food consumption by taking a picture of the food with a mobile phone and getting food annotations through a crowdsourcing system [24].”
• Im2Calories: Towards an Automated Mobile Vision Food Diary (2015, methodology): “[26] use crowdsourcing to estimate calories.”
• Examining Crowd Work and Gig Work Through the Historical Lens of Piecework (2017, methodology): “Third, system developers as on Mechanical Turk typically assume no professional skills in transcription or other areas, and attempt to build that expertise into the workflow [112, 14].”
• Online Task Assignment in Crowdsourcing Markets (2012, background): “…skills, but can result in severely suboptimal performance for human computation tasks that require specialized training or knowledge, such as analyzing the nutritional value of food, as proposed by Noronha et al. (2011), or helping people plan their travel, as proposed by Law and Zhang (2011).”
• A Practical Approach for Recognizing Eating Moments with Wrist-Mounted Inertial Sensing (2015, background): “extract nutritional information from food photographs, also through human computation [24].”
• Scalable multi-label annotation (2014, background): “For example, PlateMate [14] tags all foods in each photo for nutrition estimation.”
• No Workflow Can Ever Be Enough (2017, background): “While tasks are typically given to workers, some such as image labeling [49] and translation [29] may be automatically completed by computers or a combination of humans and machines [51, 66].”
• A Survey on Food Computing (2018, methodology): “In contrast, [Noronha et al. 2011] introduced a system PlateMate for crowdsourcing nutritional analysis from photographs of meals.”
• Crowdsourced Data Management: Industry and Academic Perspectives (2015, background): “organize crowdsourced managers and employees in the context of microtasks [139] and more complex work [118].”