Open-KNEAD: Knowledge-grounded Nutrition Estimation via Agentic Decomposition

• URL: http://arxiv.org/abs/2607.12911v1
• Date: Tue, 14 Jul 2026 15:48:25 GMT
• Title: Open-KNEAD: Knowledge-grounded Nutrition Estimation via Agentic Decomposition
• Abstract summary: We introduce Open-KNEAD, a knowledge-grounded agentic framework for meal nutrition estimation that is training-free and locally deployable.<n>Open-KNEAD improves portion estimates over both prior grounding methods and direct estimation in most backbone-dataset settings.
• Score: 9.38434232095379
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal Large Language Models (MLLMs) are increasingly used for dietary assessment from meal images, where retrieval-augmented grounding was shown to sharpen nutrition estimates. However, we find this premise no longer holds for current MLLMs. A modern MLLM's direct estimate now matches or surpasses the full retrieval pipeline. This raises a question: if retrieval no longer improves the overall estimate, can it still deliver the two things clinicians value, accurate portions and a traceable, item-by-item record? We pursue this while preserving what matters for clinical adoption: minimal user burden (a single, unannotated meal image), explainability (an auditable record), and privacy (locally hosted inference). We introduce Open-KNEAD, a knowledge-grounded agentic framework for meal nutrition estimation that is training-free and locally deployable. Each decomposed food item is grounded to a Food and Nutrient Database for Dietary Studies (FNDDS) code via selective, nutrient-aware retrieval, composing an auditable per-item record. Across two open MLLM families and three cuisines, Open-KNEAD improves portion estimates over both prior grounding methods and direct estimation in most backbone-dataset settings. An agent-internal recipe-prior step further recovers the invisible cooking-added energy that biases estimates on non-US cuisine. The advantage is largest on the dietitian-verified ACETADA dataset, where the local open agent surpasses the direct portion estimates of two frontier closed models by roughly $30\%$ and $53\%$, all while keeping every meal image on local hardware. We release the Open-KNEAD framework and its agent-ready FNDDS knowledge base.

Related papers

• OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice [4.592259709047762]
  We introduce OmniFood-Bench, a comprehensive benchmark constructed from the MM-Food-100K dataset.<n>We evaluate six state-of-the-art Large Vision-Language Models (VLMs)<n>Models achieve near-human accuracy in naming dishes, but exhibit catastrophic failure in mass estimation and frequently hallucinate benign advice for high-risk diabetic profiles.
  arXiv  Detail & Related papers  (2026-07-09T12:46:00Z)
• OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion [32.43875223470578]
  Most existing food datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes.<n>We introduce OmniFood8K, a comprehensive multimodal dataset comprising 8,036 food samples.<n>We propose an end-to-end framework for nutritional prediction from a single RGB image.
  arXiv  Detail & Related papers  (2026-04-14T06:45:32Z)
• Introducing the Swiss Food Knowledge Graph: AI for Context-Aware Nutrition Recommendation [0.0]
  We introduce the Swiss Food Knowledge Graph (SwissFKG), the first resource to unite recipes, ingredients, and their substitutions with nutrient data, dietary restrictions, allergen information, and national nutrition guidelines under one graph.<n>Our work lays the foundation for the next generation of dietary assessment tools that blend visual, contextual, and cultural dimensions of eating.
  arXiv  Detail & Related papers  (2025-07-14T11:12:30Z)
• Comprehensive Evaluation of Large Multimodal Models for Nutrition Analysis: A New Benchmark Enriched with Contextual Metadata [16.03960240895014]
  Large Multimodal Models (LMMs) are increasingly applied to meal images for nutrition analysis.<n>This work investigates how interpreting contextual metadata can enhance LMM performance in estimating key nutritional values.<n> Empirical results show that integrating metadata intelligently, when applied through straightforward prompting strategies, can significantly reduce the Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) in predicted nutritional values.
  arXiv  Detail & Related papers  (2025-07-09T17:10:33Z)
• RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models [96.43285670458803]
  Uni-Food is a unified food dataset that comprises over 100,000 images with various food labels.<n>Uni-Food is designed to provide a more holistic approach to food data analysis.<n>We introduce a novel Linear Rectification Mixture of Diverse Experts (RoDE) approach to address the inherent challenges of food-related multitasking.
  arXiv  Detail & Related papers  (2024-07-17T16:49:34Z)
• NutritionVerse-Direct: Exploring Deep Neural Networks for Multitask Nutrition Prediction from Food Images [63.314702537010355]
  Self-reporting methods are often inaccurate and suffer from substantial bias. Recent work has explored using computer vision prediction systems to predict nutritional information from food images. This paper aims to enhance the efficacy of dietary intake estimation by leveraging various neural network architectures.
  arXiv  Detail & Related papers  (2024-05-13T14:56:55Z)
• FoodLMM: A Versatile Food Assistant using Large Multi-modal Model [96.76271649854542]
  Large Multi-modal Models (LMMs) have made impressive progress in many vision-language tasks. This paper proposes FoodLMM, a versatile food assistant based on LMMs with various capabilities. We introduce a series of novel task-specific tokens and heads, enabling the model to predict food nutritional values and multiple segmentation masks.
  arXiv  Detail & Related papers  (2023-12-22T11:56:22Z)
• NutritionVerse-Real: An Open Access Manually Collected 2D Food Scene Dataset for Dietary Intake Estimation [68.49526750115429]
  We introduce NutritionVerse-Real, an open access manually collected 2D food scene dataset for dietary intake estimation. The NutritionVerse-Real dataset was created by manually collecting images of food scenes in real life, measuring the weight of every ingredient and computing the associated dietary content of each dish.
  arXiv  Detail & Related papers  (2023-11-20T11:05:20Z)
• NutritionVerse: Empirical Study of Various Dietary Intake Estimation Approaches [59.38343165508926]
  Accurate dietary intake estimation is critical for informing policies and programs to support healthy eating. Recent work has focused on using computer vision and machine learning to automatically estimate dietary intake from food images. We introduce NutritionVerse- Synth, the first large-scale dataset of 84,984 synthetic 2D food images with associated dietary information. We also collect a real image dataset, NutritionVerse-Real, containing 889 images of 251 dishes to evaluate realism.
  arXiv  Detail & Related papers  (2023-09-14T13:29:41Z)
• Towards the Creation of a Nutrition and Food Group Based Image Database [58.429385707376554]
  We propose a framework to create a nutrition and food group based image database. We design a protocol for linking food group based food codes in the U.S. Department of Agriculture's (USDA) Food and Nutrient Database for Dietary Studies (FNDDS) Our proposed method is used to build a nutrition and food group based image database including 16,114 food datasets.
  arXiv  Detail & Related papers  (2022-06-05T02:41:44Z)
• An Artificial Intelligence-Based System to Assess Nutrient Intake for Hospitalised Patients [4.048427587958764]
  Regular monitoring of nutrient intake in hospitalised patients plays a critical role in reducing the risk of disease-related malnutrition. We propose a novel system based on artificial intelligence (AI) to accurately estimate nutrient intake.
  arXiv  Detail & Related papers  (2020-03-18T15:28:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.