Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis

• URL: http://arxiv.org/abs/2507.06571v1
• Date: Wed, 09 Jul 2025 05:59:06 GMT
• Title: Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis
• Abstract summary: We propose a unified food-domain QA framework that combines a large-scale multimodal knowledge graph (MMKG) with generative AI.<n>Our MMKG links 13,000 recipes, 3,000 ingredients, 140,000 relations, and 14,000 images. We generate 40,000 QA pairs using 40 templates and LLaVA/DeepSeek augmentation.
• Score: 41.09752906121257
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We propose a unified food-domain QA framework that combines a large-scale multimodal knowledge graph (MMKG) with generative AI. Our MMKG links 13,000 recipes, 3,000 ingredients, 140,000 relations, and 14,000 images. We generate 40,000 QA pairs using 40 templates and LLaVA/DeepSeek augmentation. Joint fine-tuning of Meta LLaMA 3.1-8B and Stable Diffusion 3.5-Large improves BERTScore by 16.2\%, reduces FID by 37.8\%, and boosts CLIP alignment by 31.1\%. Diagnostic analyses-CLIP-based mismatch detection (35.2\% to 7.3\%) and LLaVA-driven hallucination checks-ensure factual and visual fidelity. A hybrid retrieval-generation strategy achieves 94.1\% accurate image reuse and 85\% adequacy in synthesis. Our results demonstrate that structured knowledge and multimodal generation together enhance reliability and diversity in food QA.

Related papers

• Enhancing Precision Agriculture with a Hybrid Deep Learning Framework for Multi-Class Plant Disease Classification and Interpretability [0.0]
  This study proposes an overall deep learning architecture for multi-class classification of plant diseases from high-resolution leaf imagery.<n>A specially gathered image database with 15,200 training images and 3,800 validation images spanned 38 classes across multiple crops, including tomato, apple, grape etc.<n>For every one of the considered architectures, ResNet-50 led to the highest accuracy of 98.74%, whereas the hybrid ResNet + ViT model achieved a competitive accuracy of 98.58%.
  arXiv  Detail & Related papers  (2026-06-13T12:41:31Z)
• P-RAG: Prompt-Enhanced Parametric RAG with LoRA and Selective CoT for Biomedical and Multi-Hop QA [9.399056753263757]
  Retrieval-Augmented Generation (RAG) addresses this constraint by retrieving external knowledge during inference.<n>We evaluate three RAG variants-Standard RAG, DA-RAG, and our proposed Prompt-Enhanced Parametric RAG (P-RAG)<n>P-RAG integrates parametric knowledge within the LLM and retrieved evidence, guided by Chain-of-Thought (CoT) prompting and Low-Rank Adaptation (LoRA)
  arXiv  Detail & Related papers  (2026-02-02T03:42:45Z)
• DrugRAG: Enhancing Pharmacy LLM Performance Through A Novel Retrieval-Augmented Generation Pipeline [4.750574899254107]
  We benchmarked eleven existing large language models (LLMs) with varying parameter sizes using a 141-question pharmacy dataset.<n>We developed a three-step retrieval-augmented generation (RAG) pipeline, DrugRAG, that retrieves structured drug knowledge from validated sources and augments model prompts with evidence-based context.<n>DrugRAG improved accuracy across all tested models, with gains ranging from 7 to 21 percentage points (e.g., Gemma 3 27B: 61% to 71%, Llama 3.1 8B: 46% to 67%) on the 141-item benchmark.
  arXiv  Detail & Related papers  (2025-12-16T20:19:23Z)
• MIRAGE: Agentic Framework for Multimodal Misinformation Detection with Web-Grounded Reasoning [0.6475163438744868]
  We present MIRAGE, an inference-time, model-pluggable agentic framework that decomposes multimodal verification into four sequential modules.<n> visual veracity assessment detects AI-generated images, cross-modal consistency analysis identifies out-of-context repurposing, retrieval-augmented factual checking grounds claims in web evidence.<n>MIRAGE orchestrates vision-language model reasoning with targeted web retrieval, outputs structured and citation-linked rationales.
  arXiv  Detail & Related papers  (2025-10-20T14:40:26Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Advanced Multi-Architecture Deep Learning Framework for BIRADS-Based Mammographic Image Retrieval: Comprehensive Performance Analysis with Super-Ensemble Optimization [0.0]
  mammographic image retrieval systems require exact BIRADS categorical matching across five distinct classes.<n>Current medical image retrieval studies suffer from methodological limitations.
  arXiv  Detail & Related papers  (2025-08-06T18:05:18Z)
• Semantic Bridge: Universal Multi-Hop Question Generation via AMR-Driven Graph Synthesis [3.1427813443719868]
  Large language model (LLM) training faces a critical bottleneck: the scarcity of high-quality, reasoning-intensive question-answer pairs.<n>We present textbfSemantic Bridge, the first universal framework for controllably generating sophisticated multi-hop reasoning questions from arbitrary sources.
  arXiv  Detail & Related papers  (2025-08-06T10:59:42Z)
• HySemRAG: A Hybrid Semantic Retrieval-Augmented Generation Framework for Automated Literature Synthesis and Methodological Gap Analysis [55.2480439325792]
  HySemRAG is a framework that combines Extract, Transform, Load (ETL) pipelines with Retrieval-Augmented Generation (RAG)<n>System addresses limitations in existing RAG architectures through a multi-layered approach.
  arXiv  Detail & Related papers  (2025-08-01T20:30:42Z)
• A Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval-Augmented Generation in Large Language Models [3.0748861313823]
  QMKGF is a Query-Aware Multi-Path Knowledge Graph Fusion Approach for Enhancing Retrieval Augmented Generation.<n>We design prompt templates and employ general-purpose LLMs to extract entities and relations.<n>We introduce a multi-path subgraph construction strategy that incorporates one-hop relations, multi-hop relations, and importance-based relations.
  arXiv  Detail & Related papers  (2025-07-07T02:22:54Z)
• MMMG: a Comprehensive and Reliable Evaluation Suite for Multitask Multimodal Generation [81.26818054877658]
  MMMG is a comprehensive benchmark for multimodal generation across 4 modality combinations.<n>It is highly aligned with human evaluation, achieving an average agreement of 94.3%.<n>GPT Image achieves 78.3% accuracy for image generation, but falls short on multimodal reasoning and interleaved generation.
  arXiv  Detail & Related papers  (2025-05-23T08:21:28Z)
• MMKB-RAG: A Multi-Modal Knowledge-Based Retrieval-Augmented Generation Framework [15.410873298893817]
  We propose Multi-Modal Knowledge-Based Retrieval-Augmented Generation (MMKB-RAG)<n>This framework leverages the inherent knowledge boundaries of models to dynamically generate semantic tags for the retrieval process.<n>Extensive experiments on knowledge-based visual question-answering tasks demonstrate the efficacy of our approach.
  arXiv  Detail & Related papers  (2025-04-14T10:19:47Z)
• KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment [1.688134675717698]
  KARMA is a novel framework employing multi-agent large language models (LLMs) to automate knowledge enrichment through structured analysis of unstructured text.<n>Our approach employs nine collaborative agents, spanning entity discovery, relation extraction, schema alignment, and conflict resolution.<n>Experiments on 1,200 PubMed articles from three different domains demonstrate the effectiveness of KARMA in knowledge graph enrichment.
  arXiv  Detail & Related papers  (2025-02-10T13:51:36Z)
• Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity [56.0251572416922]
  State Space Models (SSMs) have emerged as efficient alternatives to Transformers for sequential modeling.<n>We propose a novel SSM architecture that introduces modality-aware sparsity through modality-specific parameterization of the Mamba block.<n>We evaluate Mixture-of-Mamba across three multi-modal pretraining settings.
  arXiv  Detail & Related papers  (2025-01-27T18:35:05Z)
• MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models [115.16022378880376]
  We introduce a multimodal retrieval-augmented generation benchmark, MRAG-Bench.<n>MRAG-Bench consists of 16,130 images and 1,353 human-annotated multiple-choice questions.<n>Results show that all large vision-language models (LVLMs) exhibit greater improvements when augmented with images compared to textual knowledge.
  arXiv  Detail & Related papers  (2024-10-10T17:55:02Z)
• Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models [0.06555599394344236]
  This study evaluates the medical reasoning performance of large language models (LLMs) and vision language models (VLMs) in gastroenterology. We used 300 gastroenterology board exam-style multiple-choice questions, 138 of which contain images.
  arXiv  Detail & Related papers  (2024-08-25T14:50:47Z)
• MM-Mixing: Multi-Modal Mixing Alignment for 3D Understanding [64.65145700121442]
  We introduce MM-Mixing, a multi-modal mixing alignment framework for 3D understanding. Our proposed two-stage training pipeline combines feature-level and input-level mixing to optimize the 3D encoder. We demonstrate that MM-Mixing significantly improves baseline performance across various learning scenarios.
  arXiv  Detail & Related papers  (2024-05-28T18:44:15Z)
• Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model [41.11769935795965]
  We present a multi-stage training method combining Domain-specific Continued Pre-training (DCPT), Supervised Fine-tuning (SFT), and Direct Preference Optimization (DPO) In the CPT and SFT phases, Qilin-Med achieved 38.4% and 40.0% accuracy on the CMExam test set, respectively. In the DPO phase, it scored 16.66 in BLEU-1 and 27.44 in ROUGE-1 on the Huatuo-26M test set, bringing further improvement to the SFT phase (12.69 in BLEU-1 and 24.21 in ROUGE-1)
  arXiv  Detail & Related papers  (2023-10-13T13:17:03Z)
• An Empirical Study of Multimodal Model Merging [148.48412442848795]
  Model merging is a technique that fuses multiple models trained on different tasks to generate a multi-task solution. We conduct our study for a novel goal where we can merge vision, language, and cross-modal transformers of a modality-specific architecture. We propose two metrics that assess the distance between weights to be merged and can serve as an indicator of the merging outcomes.
  arXiv  Detail & Related papers  (2023-04-28T15:43:21Z)
• EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm [111.17100512647619]
  This paper explains the rationality of Vision Transformer by analogy with the proven practical evolutionary algorithm (EA) We propose a novel pyramid EATFormer backbone that only contains the proposed EA-based transformer (EAT) block. Massive quantitative and quantitative experiments on image classification, downstream tasks, and explanatory experiments demonstrate the effectiveness and superiority of our approach.
  arXiv  Detail & Related papers  (2022-06-19T04:49:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.