OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion

• URL: http://arxiv.org/abs/2604.12356v1
• Date: Tue, 14 Apr 2026 06:45:32 GMT
• Title: OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion
• Abstract summary: Most existing food datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes.<n>We introduce OmniFood8K, a comprehensive multimodal dataset comprising 8,036 food samples.<n>We propose an end-to-end framework for nutritional prediction from a single RGB image.
• Score: 32.43875223470578
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate estimation of food nutrition plays a vital role in promoting healthy dietary habits and personalized diet management. Most existing food datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes, which restricts accurate nutritional estimation for Chinese meals. Moreover, many state-of-the-art nutrition prediction methods rely on depth sensors, restricting their applicability in daily scenarios. To address these limitations, we introduce OmniFood8K, a comprehensive multimodal dataset comprising 8,036 food samples, each with detailed nutritional annotations and multi-view images. In addition, to enhance models' capability in nutritional prediction, we construct NutritionSynth-115K, a large-scale synthetic dataset that introduces compositional variations while preserving precise nutritional labels. Moreover, we propose an end-to-end framework for nutritional prediction from a single RGB image. First, we predict a depth map from a single RGB image and design the Scale-Shift Residual Adapter (SSRA) to refine it for global scale consistency and local structural preservation. Second, we propose the Frequency-Aligned Fusion Module (FAFM) to hierarchically align and fuse RGB and depth features in the frequency domain. Finally, we design a Mask-based Prediction Head (MPH) to emphasize key ingredient regions via dynamic channel selection for more accurate prediction. Extensive experiments on multiple datasets demonstrate the superiority of our method over existing approaches. Project homepage: https://yudongjian.github.io/OmniFood8K-food/

Related papers

• OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice [4.592259709047762]
  We introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food-100K dataset.<n>We evaluate six state-of-the-art Large Vision-Language Models (VLMs)<n>Models achieve near-human accuracy in naming dishes, but exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles.
  arXiv  Detail & Related papers  (2026-07-09T12:46:00Z)
• V-Nutri: Dish-Level Nutrition Estimation from Egocentric Cooking Videos [12.073311694888687]
  We investigate whether cooking process information from egocentric cooking videos can contribute to dish-level nutrition estimation.<n>We propose V-Nutri, a framework that combines Nutrition5K-pretrained visual backbones with a lightweight fusion module.<n> Experiments on the HD-EPIC dataset show that process cues can provide complementary nutritional evidence, improving nutrition estimation under controlled conditions.
  arXiv  Detail & Related papers  (2026-04-13T18:04:58Z)
• Real-Time Cooked Food Image Synthesis and Visual Cooking Progress Monitoring on Edge Devices [4.373318192668093]
  We introduce the first oven-based cooking-progression dataset with chef-annotated doneness levels.<n>We propose an edge-efficient recipe and cooking state guided generator that synthesizes realistic food images conditioned on raw food image.
  arXiv  Detail & Related papers  (2025-11-21T05:38:15Z)
• Advancing Food Nutrition Estimation via Visual-Ingredient Feature Fusion [69.84988999191343]
  We introduce FastFood, a dataset with 84,446 images across 908 fast food categories, featuring ingredient and nutritional annotations.<n>We propose a new model-agnostic Visual-Ingredient Feature Fusion (VIF$2$) method to enhance nutrition estimation.
  arXiv  Detail & Related papers  (2025-05-13T17:01:21Z)
• NutritionVerse-Synth: An Open Access Synthetically Generated 2D Food Scene Dataset for Dietary Intake Estimation [71.22646949733833]
  We introduce NutritionVerse- Synth (NV- Synth), a large-scale synthetic food image dataset. NV- Synth contains 84,984 photorealistic meal images rendered from 7,082 dynamically plated 3D scenes. As the largest open-source synthetic food dataset, NV- Synth highlights the value of physics-based simulations.
  arXiv  Detail & Related papers  (2023-12-11T08:15:49Z)
• NutritionVerse-Real: An Open Access Manually Collected 2D Food Scene Dataset for Dietary Intake Estimation [68.49526750115429]
  We introduce NutritionVerse-Real, an open access manually collected 2D food scene dataset for dietary intake estimation. The NutritionVerse-Real dataset was created by manually collecting images of food scenes in real life, measuring the weight of every ingredient and computing the associated dietary content of each dish.
  arXiv  Detail & Related papers  (2023-11-20T11:05:20Z)
• NutritionVerse: Empirical Study of Various Dietary Intake Estimation Approaches [59.38343165508926]
  Accurate dietary intake estimation is critical for informing policies and programs to support healthy eating. Recent work has focused on using computer vision and machine learning to automatically estimate dietary intake from food images. We introduce NutritionVerse- Synth, the first large-scale dataset of 84,984 synthetic 2D food images with associated dietary information. We also collect a real image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to evaluate realism.
  arXiv  Detail & Related papers  (2023-09-14T13:29:41Z)
• UMDFood: Vision-language models boost food composition compilation [26.5694236976957]
  We propose a novel vision-language model, UMDFood-VL, using front-of-package labeling and product images to accurately estimate food composition profiles. Up to 82.2% of selected products' estimated error between chemical analysis results and model estimation results are less than 10%. This performance sheds light on generalization towards other food and nutrition-related data compilation and catalyzation.
  arXiv  Detail & Related papers  (2023-05-18T03:18:12Z)
• Towards the Creation of a Nutrition and Food Group Based Image Database [58.429385707376554]
  We propose a framework to create a nutrition and food group based image database. We design a protocol for linking food group based food codes in the U.S. Department of Agriculture's (USDA) Food and Nutrient Database for Dietary Studies (FNDDS) Our proposed method is used to build a nutrition and food group based image database including 16,114 food datasets.
  arXiv  Detail & Related papers  (2022-06-05T02:41:44Z)
• An End-to-End Food Image Analysis System [8.622335099019214]
  We propose an image-based food analysis framework that integrates food localization, classification and portion size estimation. Our proposed framework is end-to-end, i.e., the input can be an arbitrary food image containing multiple food items. Our framework is evaluated on a real life food image dataset collected from a nutrition feeding study.
  arXiv  Detail & Related papers  (2021-02-01T05:36:20Z)
• ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network [50.7720194859196]
  We introduce the dataset ISIA Food- 500 with 500 categories from the list in the Wikipedia and 399,726 images. This dataset surpasses existing popular benchmark datasets by category coverage and data volume. We propose a stacked global-local attention network, which consists of two sub-networks for food recognition.
  arXiv  Detail & Related papers  (2020-08-13T02:48:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.