CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

• URL: http://arxiv.org/abs/2609.03526v1
• Date: Thu, 03 Sep 2026 08:24:22 GMT
• Title: CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning
• Abstract summary: CulturalMenuBench is a benchmark of 4,870 items in 10 languages across 18 regions.<n>Its 10 tasks pair final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels.<n>Models exceeding 94% on standard multiple-choice tasks drop to at most 56% when attributing dishes to Chinese regional cuisines.
• Score: 54.76869488340919
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cultural understanding or mere visual matching. To probe this distinction, we introduce CulturalMenuBench, a benchmark of 4,870 items in 10 languages across 18 regions; its 10 tasks pair final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels, spanning basic recognition to process-grounded cultural attribution. Evaluating 12 models exposes a substantial knowledge-application gap: models exceeding 94% on standard multiple-choice tasks drop to at most 56% when attributing dishes to Chinese regional cuisines, despite an identical four-way format. Diagnostic analyses explain why: error patterns are consistent with random guessing, accuracy tracks visual distinctiveness rather than cultural structure, and models classify cuisines more accurately from dish names alone than from images (+7-18 points). The knowledge is thus present but cannot be activated through visual input. An ablation confirms these tasks genuinely require procedural evidence: removing sequential cooking images selectively degrades process-grounded tasks while others remain stable. Overall, CulturalMenuBench shows that near-perfect recognition can conceal an inability to apply cultural knowledge, motivating training that explicitly connects perception, procedure, and cultural context. Code and data are publicly available.

Related papers

• Right Frame, Wrong Rule: Cultural Cues Expose the Financial Knowledge Gap They Were Meant to Close [57.963463622270275]
  We study normative pluralism in Islamic finance using a four-choice taxonomy.<n>A cultural cue steers a model toward one framework, but the model selects an incorrect answer within that framework.
  arXiv  Detail & Related papers  (2026-09-01T09:46:53Z)
• World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models [41.385606397781714]
  We investigate how Large Vision-Language Models perceive culture mixing scenarios.<n>We use CultureMix, a food Visual Question Answering (VQA) benchmark with 23k diffusion-generated, human-verified culture mixing images.<n>We find consistent failures to preserve individual cultural identities in mixed settings.
  arXiv  Detail & Related papers  (2025-11-27T22:23:08Z)
• Mitigating Cross-modal Representation Bias for Multicultural Image-to-Recipe Retrieval [33.17028372962136]
  Cross-modal representations to bridge the modality gap between images and recipes tend to ignore subtle recipe-specific details.<n>This paper proposes a novel causal approach that predicts the culinary elements potentially overlooked in images.<n> Experiments are conducted on the standard monolingual Recipe1M dataset and a newly curated multilingual multicultural cuisine dataset.
  arXiv  Detail & Related papers  (2025-10-23T09:43:43Z)
• When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts [15.78054683369659]
  We introduce MixCuBe, a cross-cultural bias benchmark, and study elements from five countries and four ethnicities.<n>Our findings reveal that MLLMs achieve both higher accuracy and lower sensitivity to such perturbations for high-resource cultures.<n>GPT-4o, the best-performing model overall, shows up to 58% difference in accuracy between the original and perturbed cultural settings in low-resource cultures.
  arXiv  Detail & Related papers  (2025-03-21T03:50:05Z)
• WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines [74.25764182510295]
  Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English.<n>We introduce World Cuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding.<n>This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points.
  arXiv  Detail & Related papers  (2024-10-16T16:11:49Z)
• CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming [75.82306181299153]
  CulturalBench is a set of 1,696 human-written and human-verified questions to assess LMs' cultural knowledge.<n>It covers 45 global regions including underrepresented ones like Bangladesh, Zimbabwe, and Peru.<n>We construct CulturalBench using methods inspired by Human-AI Red-Teaming.
  arXiv  Detail & Related papers  (2024-10-03T17:04:31Z)
• Crossroads of Continents: Automated Artifact Extraction for Cultural Adaptation with Large Multimodal Models [22.92083941222383]
  We introduce DalleStreet, a large-scale dataset generated by DALL-E 3 and validated by humans. We find disparities in cultural understanding at geographic sub-region levels with both open-source (LLaVA) and closed-source (GPT-4V) models. Our findings reveal a nuanced picture of the cultural competence of LMMs, highlighting the need to develop culture-aware systems.
  arXiv  Detail & Related papers  (2024-07-02T08:55:41Z)
• FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture [60.51749998013166]
  We introduce FoodieQA, a manually curated, fine-grained image-text dataset capturing the intricate features of food cultures across various regions in China. We evaluate vision-language Models (VLMs) and large language models (LLMs) on newly collected, unseen food images and corresponding questions. Our findings highlight that understanding food and its cultural implications remains a challenging and under-explored direction.
  arXiv  Detail & Related papers  (2024-06-16T17:59:32Z)
• CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark [68.21939124278065]
  Culturally-diverse multilingual Visual Question Answering benchmark designed to cover a rich set of languages and cultures. CVQA includes culturally-driven images and questions from across 30 countries on four continents, covering 31 languages with 13 scripts, providing a total of 10k questions. We benchmark several Multimodal Large Language Models (MLLMs) on CVQA, and show that the dataset is challenging for the current state-of-the-art models.
  arXiv  Detail & Related papers  (2024-06-10T01:59:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.