CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

• URL: http://arxiv.org/abs/2607.25294v1
• Date: Tue, 28 Jul 2026 05:06:43 GMT
• Title: CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition
• Abstract summary: We introduce CLBench-V, a benchmark for multimodal context learning.<n> CLBench-V organizes tasks around three dimensions: context grounding, new information application, and new knowledge learning.<n>Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847.
• Score: 12.300125680559338
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Real-world tasks often require models to learn from task-specific context rather than relying only on pre-trained knowledge. While recent work has highlighted this capability as context learning, existing evaluations mainly focus on textual contexts. In many practical settings, however, the context to be learned from is multimodal: scientific findings are conveyed through figures and tables, financial indicators are scattered across converted reports, and spatial decisions depend on maps, scenes, or web pages. We introduce CLBench-V, a benchmark for multimodal context learning that addresses the difficulty of localizing where context use breaks down by organizing tasks around three dimensions: context grounding, new information application, and new knowledge learning. CLBench-V combines converted public benchmarks with newly constructed datasets spanning domains such as science, finance, long-document understanding, spatial reasoning, and web-based visual question answering. To reduce the cost of constructing domain-specific context-learning tasks, we further use automated construction and filtering procedures for our newly built datasets. Across 3,443 instances and six recent multimodal models, the best overall score is only 0.2847, indicating that multimodal context learning remains far from saturated. Moreover, InternVL3.5-30B-A3B performs best on context grounding and new knowledge learning, while Qwen3.5-Plus performs best on new information application. We further analyze judge reliability, context length, image count, and representative failure cases. Code is available at https://github.com/IamLihua/CLBench-V.

Related papers

• Deeply Interleaved Text-Image Contexts for Multimodal LLMs Assessment [57.41731607488101]
  TIC-Bench is designed to evaluate the capability of models to integrate text-image clues and recover the ground truth facts within deeply interleaved contexts.<n>We evaluated 10 state-of-the-art MLLMs and observed a substantial performance gap compared to human experts.
  arXiv  Detail & Related papers  (2026-09-02T13:19:51Z)
• Context Training with Active Information Seeking [23.67605567495757]
  We equip contexts with Wikipedia search and browser tools for active information seeking.<n>We show that when paired with a search-based training procedure that maintains and prunes multiple candidate contexts, active information seeking delivers consistent and substantial gains.
  arXiv  Detail & Related papers  (2026-05-13T06:15:32Z)
• MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence [66.12781312894647]
  MMCL-Bench is a benchmark for learning task-local rules, procedures, and empirical patterns from visual or mixed-modality teaching context.<n>This setting requires models to recover and localize relevant evidence from images, screenshots, manuals, videos, and frame sequences before they can reason over the learned context.<n>We evaluate frontier multimodal models with strict-based scoring and find that current systems remain far from robust multimodal context learning.
  arXiv  Detail & Related papers  (2026-05-12T19:57:37Z)
• CL-bench: A Benchmark for Context Learning [152.2879060355882]
  We term this capability context learning, a crucial ability that humans naturally possess but has been largely overlooked.<n>We introduce CL-bench, a real-world benchmark consisting of 500 contexts, 1,899 tasks, and 31,607 verifications.<n> CL-bench represents a step towards building LMs with this fundamental capability, making them more intelligent and advancing their deployment in real-world scenarios.
  arXiv  Detail & Related papers  (2026-02-03T14:37:47Z)
• MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs [61.56904387052982]
  This paper proposes a new visual grounding task called multi-context visual grounding.<n>It aims to localize instances of interest across multiple images based on open-ended text prompts.<n>We benchmark over 20 state-of-the-art MLLMs and foundation models with potential multi-context visual grounding capabilities.
  arXiv  Detail & Related papers  (2024-10-16T07:52:57Z)
• BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data [61.936320820180875]
  Large language models (LLMs) have become increasingly pivotal across various domains. BabelBench is an innovative benchmark framework that evaluates the proficiency of LLMs in managing multimodal multistructured data with code execution. Our experimental findings on BabelBench indicate that even cutting-edge models like ChatGPT 4 exhibit substantial room for improvement.
  arXiv  Detail & Related papers  (2024-10-01T15:11:24Z)
• NeedleBench: Evaluating LLM Retrieval and Reasoning Across Varying Information Densities [51.07379913779232]
  NeedleBench is a framework for assessing retrieval and reasoning performance in long-context tasks.<n>It embeds key data points at varying depths to rigorously test model capabilities.<n>Our experiments reveal that reasoning models like Deep-R1 and OpenAI's o3 struggle with continuous retrieval and reasoning in information-dense scenarios.
  arXiv  Detail & Related papers  (2024-07-16T17:59:06Z)
• M3: A Multi-Task Mixed-Objective Learning Framework for Open-Domain Multi-Hop Dense Sentence Retrieval [12.277521531556852]
  M3 is a novel Multi-hop dense sentence retrieval system built upon a novel Multi-task Mixed-objective approach for dense text representation learning. Our approach yields state-of-the-art performance on a large-scale open-domain fact verification benchmark dataset, FEVER.
  arXiv  Detail & Related papers  (2024-03-21T01:52:07Z)
• Generative Multi-Modal Knowledge Retrieval with Large Language Models [75.70313858231833]
  We propose an innovative end-to-end generative framework for multi-modal knowledge retrieval. Our framework takes advantage of the fact that large language models (LLMs) can effectively serve as virtual knowledge bases. We demonstrate significant improvements ranging from 3.0% to 14.6% across all evaluation metrics when compared to strong baselines.
  arXiv  Detail & Related papers  (2024-01-16T08:44:29Z)
• Few-shot Multimodal Multitask Multilingual Learning [0.0]
  We propose few-shot learning for a multimodal multitask multilingual (FM3) setting by adapting pre-trained vision and language models. FM3 learns the most prominent tasks in the vision and language domains along with their intersections.
  arXiv  Detail & Related papers  (2023-02-19T03:48:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.