MetaFood3D: 3D Food Dataset with Nutrition Values

• URL: http://arxiv.org/abs/2409.01966v2
• Date: Sat, 07 Dec 2024 21:37:07 GMT
• Title: MetaFood3D: 3D Food Dataset with Nutrition Values
• Authors: Yuhao Chen, Jiangpeng He, Gautham Vinod, Siddeshwar Raghavan, Chris Czarnecki, Jinge Ma, Talha Ibn Mahmud, Bruce Coburn, Dayou Mao, Saeejith Nair, Pengcheng Xi, Alexander Wong, Edward Delp, Fengqing Zhu,
• Abstract summary: This dataset consists of 743 meticulously scanned and labeled 3D food objects across 131 categories. Our MetaFood3D dataset emphasizes intra-class diversity and includes rich modalities such as textured mesh files, RGB-D videos, and segmentation masks.
• Score: 52.16894900096017
• License:
• Abstract: Food computing is both important and challenging in computer vision (CV). It significantly contributes to the development of CV algorithms due to its frequent presence in datasets across various applications, ranging from classification and instance segmentation to 3D reconstruction. The polymorphic shapes and textures of food, coupled with high variation in forms and vast multimodal information, including language descriptions and nutritional data, make food computing a complex and demanding task for modern CV algorithms. 3D food modeling is a new frontier for addressing food related problems, due to its inherent capability to deal with random camera views and its straightforward representation for calculating food portion size. However, the primary hurdle in the development of algorithms for food object analysis is the lack of nutrition values in existing 3D datasets. Moreover, in the broader field of 3D research, there is a critical need for domain-specific test datasets. To bridge the gap between general 3D vision and food computing research, we introduce MetaFood3D. This dataset consists of 743 meticulously scanned and labeled 3D food objects across 131 categories, featuring detailed nutrition information, weight, and food codes linked to a comprehensive nutrition database. Our MetaFood3D dataset emphasizes intra-class diversity and includes rich modalities such as textured mesh files, RGB-D videos, and segmentation masks. Experimental results demonstrate our dataset's strong capabilities in enhancing food portion estimation algorithms, highlight the gap between video captures and 3D scanned data, and showcase the strengths of MetaFood3D in generating synthetic eating occasion data and 3D food objects.

Related papers

• MFP3D: Monocular Food Portion Estimation Leveraging 3D Point Clouds [7.357322789192671]
  In this paper, we introduce a new framework for accurate food estimation using only a single monocular image. The framework consists of three key modules: (1) a 3D Reconstruction Module that generates a 3D point cloud representation of the food from the 2D image, (2) a Feature Extraction Module that extracts and represents features from both the 3D point cloud and the 2D RGB image, and (3) a Portion Regression Module that employs a deep regression model to estimate the food's volume and energy content.
  arXiv  Detail & Related papers  (2024-11-14T22:17:27Z)
• MetaFood CVPR 2024 Challenge on Physically Informed 3D Food Reconstruction: Methods and Results [52.07174491056479]
  We host the MetaFood Workshop and its challenge for Physically Informed 3D Food Reconstruction. This challenge focuses on reconstructing volume-accurate 3D models of food items from 2D images, using a visible checkerboard as a size reference. The solutions developed in this challenge achieved promising results in 3D food reconstruction, with significant potential for improving portion estimation for dietary assessment and nutritional monitoring.
  arXiv  Detail & Related papers  (2024-07-12T14:15:48Z)
• Food Portion Estimation via 3D Object Scaling [8.164262056488447]
  We propose a new framework to estimate both food volume and energy from 2D images. Our method estimates the pose of the camera and the food object in the input image. We also introduce a new dataset, SimpleFood45, which contains 2D images of 45 food items.
  arXiv  Detail & Related papers  (2024-04-18T15:23:37Z)
• NutritionVerse-Synth: An Open Access Synthetically Generated 2D Food Scene Dataset for Dietary Intake Estimation [71.22646949733833]
  We introduce NutritionVerse- Synth (NV- Synth), a large-scale synthetic food image dataset. NV- Synth contains 84,984 photorealistic meal images rendered from 7,082 dynamically plated 3D scenes. As the largest open-source synthetic food dataset, NV- Synth highlights the value of physics-based simulations.
  arXiv  Detail & Related papers  (2023-12-11T08:15:49Z)
• Transferring Knowledge for Food Image Segmentation using Transformers and Convolutions [65.50975507723827]
  Food image segmentation is an important task that has ubiquitous applications, such as estimating the nutritional value of a plate of food. One challenge is that food items can overlap and mix, making them difficult to distinguish. Two models are trained and compared, one based on convolutional neural networks and the other on Bidirectional representation for Image Transformers (BEiT) The BEiT model outperforms the previous state-of-the-art model by achieving a mean intersection over union of 49.4 on FoodSeg103.
  arXiv  Detail & Related papers  (2023-06-15T15:38:10Z)
• NutritionVerse-3D: A 3D Food Model Dataset for Nutritional Intake Estimation [65.47310907481042]
  One in four older adults are malnourished. Machine learning and computer vision show promise of automated nutrition tracking methods of food. NutritionVerse-3D is a large-scale high-resolution dataset of 105 3D food models.
  arXiv  Detail & Related papers  (2023-04-12T05:27:30Z)
• Towards Building a Food Knowledge Graph for Internet of Food [66.57235827087092]
  We review the evolution of food knowledge organization, from food classification to food to food knowledge graphs. Food knowledge graphs play an important role in food search and Question Answering (QA), personalized dietary recommendation, food analysis and visualization. Future directions for food knowledge graphs cover several fields such as multimodal food knowledge graphs and food intelligence.
  arXiv  Detail & Related papers  (2021-07-13T06:26:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.