VolETA: One- and Few-shot Food Volume Estimation

• URL: http://arxiv.org/abs/2407.01717v1
• Date: Mon, 1 Jul 2024 18:47:15 GMT
• Title: VolETA: One- and Few-shot Food Volume Estimation
• Authors: Ahmad AlMughrabi, Umair Haroon, Ricardo Marques, Petia Radeva,
• Abstract summary: We present VolETA, a sophisticated methodology for estimating food volume using 3D generative techniques. Our approach creates a scaled 3D mesh of food objects using one- or few-RGBD images. We achieve robust and accurate volume estimations with 10.97% MAPE using the MTF dataset.
• Score: 4.282795945742752
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Accurate food volume estimation is essential for dietary assessment, nutritional tracking, and portion control applications. We present VolETA, a sophisticated methodology for estimating food volume using 3D generative techniques. Our approach creates a scaled 3D mesh of food objects using one- or few-RGBD images. We start by selecting keyframes based on the RGB images and then segmenting the reference object in the RGB images using XMem++. Simultaneously, camera positions are estimated and refined using the PixSfM technique. The segmented food images, reference objects, and camera poses are combined to form a data model suitable for NeuS2. Independent mesh reconstructions for reference and food objects are carried out, with scaling factors determined using MeshLab based on the reference object. Moreover, depth information is used to fine-tune the scaling factors by estimating the potential volume range. The fine-tuned scaling factors are then applied to the cleaned food meshes for accurate volume measurements. Similarly, we enter a segmented RGB image to the One-2-3-45 model for one-shot food volume estimation, resulting in a mesh. We then leverage the obtained scaling factors to the cleaned food mesh for accurate volume measurements. Our experiments show that our method effectively addresses occlusions, varying lighting conditions, and complex food geometries, achieving robust and accurate volume estimations with 10.97% MAPE using the MTF dataset. This innovative approach enhances the precision of volume assessments and significantly contributes to computational nutrition and dietary monitoring advancements.

Related papers

• KRONC: Keypoint-based Robust Camera Optimization for 3D Car Reconstruction [58.04846444985808]
  This paper introduces KRONC, a novel approach aimed at inferring view poses by leveraging prior knowledge about the object to reconstruct and its representation through semantic keypoints. With a focus on vehicle scenes, KRONC is able to estimate the position of the views as a solution to a light optimization problem targeting the convergence of keypoints' back-projections to a singular point.
  arXiv  Detail & Related papers  (2024-09-09T08:08:05Z)
• MetaFood3D: Large 3D Food Object Dataset with Nutrition Values [53.24500333363066]
  This dataset consists of 637 meticulously labeled 3D food objects across 108 categories, featuring detailed nutrition information, weight, and food codes linked to a comprehensive nutrition database. Experimental results demonstrate our dataset's significant potential for improving algorithm performance, highlight the challenging gap between video captures and 3D scanned data, and show the strength of the MetaFood3D dataset in high-quality data generation, simulation, and augmentation.
  arXiv  Detail & Related papers  (2024-09-03T15:02:52Z)
• Vision-Based Approach for Food Weight Estimation from 2D Images [0.9208007322096533]
  The study employs a dataset of 2380 images comprising fourteen different food types in various portions, orientations, and containers. The proposed methodology integrates deep learning and computer vision techniques, specifically employing Faster R-CNN for food detection and MobileNetV3 for weight estimation.
  arXiv  Detail & Related papers  (2024-05-26T08:03:51Z)
• Food Portion Estimation via 3D Object Scaling [8.164262056488447]
  We propose a new framework to estimate both food volume and energy from 2D images. Our method estimates the pose of the camera and the food object in the input image. We also introduce a new dataset, SimpleFood45, which contains 2D images of 45 food items.
  arXiv  Detail & Related papers  (2024-04-18T15:23:37Z)
• An End-to-end Food Portion Estimation Framework Based on Shape Reconstruction from Monocular Image [7.380382380564532]
  We propose an end-to-end deep learning framework for food energy estimation from a monocular image through 3D shape reconstruction. Our method is evaluated on a publicly available food image dataset Nutrition5k, resulting a Mean Absolute Error (MAE) of 40.05 kCal and Mean Absolute Percentage Error (MAPE) of 11.47% for food energy estimation.
  arXiv  Detail & Related papers  (2023-08-03T15:17:24Z)
• Transferring Knowledge for Food Image Segmentation using Transformers and Convolutions [65.50975507723827]
  Food image segmentation is an important task that has ubiquitous applications, such as estimating the nutritional value of a plate of food. One challenge is that food items can overlap and mix, making them difficult to distinguish. Two models are trained and compared, one based on convolutional neural networks and the other on Bidirectional representation for Image Transformers (BEiT) The BEiT model outperforms the previous state-of-the-art model by achieving a mean intersection over union of 49.4 on FoodSeg103.
  arXiv  Detail & Related papers  (2023-06-15T15:38:10Z)
• Category-Level Metric Scale Object Shape and Pose Estimation [73.92460712829188]
  We propose a framework that jointly estimates a metric scale shape and pose from a single RGB image. We validated our method on both synthetic and real-world datasets to evaluate category-level object pose and shape.
  arXiv  Detail & Related papers  (2021-09-01T12:16:46Z)
• DONet: Learning Category-Level 6D Object Pose and Size Estimation from Depth Observation [53.55300278592281]
  We propose a method of Category-level 6D Object Pose and Size Estimation (COPSE) from a single depth image. Our framework makes inferences based on the rich geometric information of the object in the depth channel alone. Our framework competes with state-of-the-art approaches that require labeled real-world images.
  arXiv  Detail & Related papers  (2021-06-27T10:41:50Z)
• A Large-Scale Benchmark for Food Image Segmentation [62.28029856051079]
  We build a new food image dataset FoodSeg103 (and its extension FoodSeg154) containing 9,490 images. We annotate these images with 154 ingredient classes and each image has an average of 6 ingredient labels and pixel-wise masks. We propose a multi-modality pre-training approach called ReLeM that explicitly equips a segmentation model with rich and semantic food knowledge.
  arXiv  Detail & Related papers  (2021-05-12T03:00:07Z)
• An End-to-End Food Image Analysis System [8.622335099019214]
  We propose an image-based food analysis framework that integrates food localization, classification and portion size estimation. Our proposed framework is end-to-end, i.e., the input can be an arbitrary food image containing multiple food items. Our framework is evaluated on a real life food image dataset collected from a nutrition feeding study.
  arXiv  Detail & Related papers  (2021-02-01T05:36:20Z)
• Multi-Task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation [6.603050343996914]
  We propose an end-to-end multi-task framework that can achieve both food classification and food portion size estimation. Our results outperforms the baseline methods for both classification accuracy and mean absolute error for portion estimation.
  arXiv  Detail & Related papers  (2020-04-27T21:35:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.