MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence

• URL: http://arxiv.org/abs/2605.12703v1
• Date: Tue, 12 May 2026 19:57:37 GMT
• Title: MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence
• Abstract summary: MMCL-Bench is a benchmark for learning task-local rules, procedures, and empirical patterns from visual or mixed-modality teaching context.<n>This setting requires models to recover and localize relevant evidence from images, screenshots, manuals, videos, and frame sequences before they can reason over the learned context.<n>We evaluate frontier multimodal models with strict-based scoring and find that current systems remain far from robust multimodal context learning.
• Score: 66.12781312894647
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mixed-modality teaching context and applying them to new visual instances. Unlike text-only context learning or standard multimodal question answering, this setting requires models to recover and localize relevant evidence from images, screenshots, manuals, videos, and frame sequences before they can reason over the learned context. MMCL-Bench contains 102 tasks spanning three categories: rule system application, procedural task execution, and empirical discovery and induction. We evaluate frontier multimodal models with strict rubric-based scoring and find that current systems remain far from robust multimodal context learning, with even the strongest model solving fewer than one-third of tasks under strict evaluation. Diagnostic ablations and error analysis show that failures arise throughout the context-to-answer pipeline, including context anchoring, visual evidence extraction, context reasoning, and response construction. MMCL-Bench thus highlights multimodal context learning as an important unsolved capability bottleneck for current multimodal models.

Related papers

• CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition [12.300125680559338]
  We introduce CLBench-V, a benchmark for multimodal context learning.<n> CLBench-V organizes tasks around three dimensions: context grounding, new information application, and new knowledge learning.<n>Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847.
  arXiv  Detail & Related papers  (2026-07-28T05:06:43Z)
• Enhancing Multimodal In-Context Learning via Inductive-Deductive Reasoning [65.15766304205657]
  In-context learning (ICL) allows large models to adapt to tasks using a few examples, yet its extension to vision-language models (VLMs) remains fragile.<n>Our analysis reveals that the fundamental limitation lies in an inductive gap, models often produce correct answers from flawed reasoning.<n>We introduce a framework that restructures multimodal ICL as a principled inductive-deductive process.
  arXiv  Detail & Related papers  (2026-05-04T09:18:19Z)
• LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations [53.20772659095155]
  This paper tackles the more challenging setting of IML under training-time incomplete observations.<n>We propose LIMSSR (LLM-Driven Incomplete Multimodal Sequence-to-Score Reasoning), a framework that reformulates this challenge as a conditional sequence reasoning task.
  arXiv  Detail & Related papers  (2026-05-01T06:11:42Z)
• Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks [7.62772056485722]
  In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations.<n>Despite its success in large language models, the extension of ICL to multimodal settings remains poorly understood.<n>We show that multimodal ICL performs comparably to text-only ICL in zero-shot settings but degrades significantly under few-shot demonstrations.
  arXiv  Detail & Related papers  (2026-04-15T02:12:51Z)
• What do vision-language models see in the context? Investigating multimodal in-context learning [2.1119217917006234]
  In-context learning (ICL) enables Large Language Models to learn tasks from demonstration examples without parameter updates.<n>We present a systematic study of ICL in Vision-Language Models (VLMs)<n>We analyze how prompt design, architectural choices, and training strategies influence multimodal ICL.
  arXiv  Detail & Related papers  (2025-10-28T11:55:24Z)
• ContextNav: Towards Agentic Multimodal In-Context Learning [85.05420047017513]
  ContextNav is an agentic framework that integrates the scalability of automated retrieval with the quality and adaptiveness of human-like curation.<n>It builds a resource-aware multimodal embedding pipeline, maintains a retrievable vector database, and applies agentic retrieval and structural alignment to construct noise-resilient contexts.<n> Experimental results demonstrate that ContextNav achieves state-of-the-art performance across various datasets.
  arXiv  Detail & Related papers  (2025-10-06T07:49:52Z)
• True Multimodal In-Context Learning Needs Attention to the Visual Context [69.63677595066012]
  Multimodal Large Language Models (MLLMs) have enabled Multimodal In-Context Learning (MICL)-adapting to new tasks.<n>Current MLLMs tend to neglect visual cues and over-rely on textual patterns, leading to mere text imitation rather than genuine multimodal adaptation.<n>We introduce Dynamic Attention Reallocation (DARA), an efficient fine-tuning strategy that encourages models to attend to the visual context.
  arXiv  Detail & Related papers  (2025-07-21T17:08:18Z)
• Towards More Unified In-context Visual Understanding [74.55332581979292]
  We present a new ICL framework for visual understanding with multi-modal output enabled. First, we quantize and embed both text and visual prompt into a unified representational space. Then a decoder-only sparse transformer architecture is employed to perform generative modeling on them.
  arXiv  Detail & Related papers  (2023-12-05T06:02:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.