MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning

• URL: http://arxiv.org/abs/2603.12266v1
• Date: Thu, 12 Mar 2026 17:59:56 GMT
• Title: MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning
• Abstract summary: We introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning.<n> MM-CondChain is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition.<n>A Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable.
• Score: 37.02901476078596
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to carry out visual workflows such as navigating GUIs, where the next step depends on verified visual compositional conditions (e.g., "if a permission dialog appears and the color of the interface is green, click Allow") and the process may branch or terminate early. Yet this capability remains under-evaluated: existing benchmarks focus on shallow-compositions or independent-constraints rather than deeply chained compositional conditionals. In this paper, we introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning. Each benchmark instance is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition grounded in visual evidence and built from multiple objects, attributes, or relations. To answer correctly, an MLLM must perceive the image in detail, reason over multiple visual elements at each step, and follow the resulting execution path to the final outcome. To scalably construct such workflow-style data, we propose an agentic synthesis pipeline: a Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable. A Composer then assembles these verified layers into complete instructions. Using this pipeline, we construct benchmarks across three visual domains: natural images, data charts, and GUI trajectories. Experiments on a range of MLLMs show that even the strongest model attains only 53.33 Path F1, with sharp drops on hard negatives and as depth or predicate complexity grows, confirming that deep compositional reasoning remains a fundamental challenge.

Related papers

• Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning [40.55876706740581]
  Explicit visual intermediates can help multimodal large language models externalize spatial evidence and updated visual states.<n>We introduce textbfAphanta, an automated task-discovery and closed-loop diagnostic framework for the MLLM -> image editor -> MLLM pipeline.<n>Aphanta evaluates three conditions---direct reasoning, reasoning with an editor-generated intermediate, and reasoning with an idealized reference intermediate.
  arXiv  Detail & Related papers  (2026-08-27T11:41:15Z)
• Deep Residual Injection for Full-Spectrum Forensic Signal Perception in Multimodal Large Language Models [51.099179436350376]
  Multimodal large language models (MLLMs) have been increasingly adopted in forensics for their robust semantic understanding.<n>We propose Deep Visual Residual MLLM (Deep-VRM) to preserve early semantic processing while injecting artifact-specific visual signals as a residual path into an intermediate layer.<n>This enables later layers to jointly model semantic reasoning and signal-level forensic cues, and surprisingly, the model learns to adaptively leverage different levels of forensic signals depending on the input.
  arXiv  Detail & Related papers  (2026-06-14T16:05:45Z)
• GALA: Multimodal Graph Alignment for Bug Localization in Automated Program Repair [40.228155198574356]
  We propose GALA, a framework that shifts multimodal Automated Program Repair (APR) from implicit semantic guessing to explicit structural reasoning.<n>GALA operates in four stages: it first constructs an Image UI Graph to capture visual elements and their structural relationships; then performs file-level alignment by cross-referencing this UI graph with repository-level structures.<n>By enforcing both semantic and relational consistency across modalities, GALA establishes a highly accurate visual-to-code mapping.
  arXiv  Detail & Related papers  (2026-04-09T11:06:25Z)
• Beyond Global Similarity: Towards Fine-Grained, Multi-Condition Multimodal Retrieval [27.493644447594367]
  MCMR (Multi-Conditional Multimodal Retrieval) is a large-scale benchmark designed to evaluate fine-grained, multi-condition cross-modal retrieval under natural-language queries.<n>It spans five product domains: upper and bottom clothing, jewelry, shoes, and furniture.<n>We benchmark a diverse suite of MLLM-based multimodal retrievers and vision-language rerankers to assess their condition-aware reasoning abilities.
  arXiv  Detail & Related papers  (2026-03-01T12:53:47Z)
• Referring Layer Decomposition [25.128453386102887]
  We introduce the Referring Layer Decomposition (RLD) task, which predicts complete RGBA layers from a single RGB image.<n>At the core is the RefLade, a large-scale dataset comprising 1.11M image-layer-prompt triplets produced by our scalable data engine.<n>We present RefLayer, a simple baseline designed for prompt-conditioned layer decomposition, achieving high visual fidelity and semantic alignment.
  arXiv  Detail & Related papers  (2026-02-22T22:05:17Z)
• Resolving Evidence Sparsity: Agentic Context Engineering for Long-Document Understanding [49.26132236798123]
  Vision Language Models (VLMs) have gradually become a primary approach in document understanding.<n>We propose SLEUTH, a multi agent framework that orchestrates a retriever and four collaborative agents in a coarse to fine process.<n>The framework identifies key textual and visual clues within the retrieved pages, filters for salient visual evidence such as tables and charts, and analyzes the query to devise a reasoning strategy.
  arXiv  Detail & Related papers  (2025-11-28T03:09:40Z)
• CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation [54.53371540755023]
  Reasoning segmentation seeks pixel-accurate masks for targets referenced by complex, often implicit instructions.<n>We present CORA, a semi-supervised reasoning segmentation framework that jointly learns from limited labeled data and a large corpus of unlabeled images.<n> CORA achieves state-of-the-art results, requiring as few as 100 labeled images on Cityscapes, a benchmark dataset for urban scene understanding.
  arXiv  Detail & Related papers  (2025-11-21T20:14:55Z)
• How Multimodal LLMs Solve Image Tasks: A Lens on Visual Grounding, Task Reasoning, and Answer Decoding [39.342366994703376]
  We introduce a probing framework to analyze how MLLMs process visual and textual inputs across layers.<n>We show that while the overall stage-wise structure remains stable across variations in visual tokenization, instruction tuning data, and pretraining corpus, the specific layer allocation to each stage shifts.
  arXiv  Detail & Related papers  (2025-08-27T21:22:01Z)
• Towards Text-Image Interleaved Retrieval [49.96332254241075]
  We introduce the text-image interleaved retrieval (TIIR) task, where the query and document are interleaved text-image sequences.<n>We construct a TIIR benchmark based on naturally interleaved wikiHow tutorials, where a specific pipeline is designed to generate interleaved queries.<n>We propose a novel Matryoshka Multimodal Embedder (MME), which compresses the number of visual tokens at different granularity.
  arXiv  Detail & Related papers  (2025-02-18T12:00:47Z)
• Automated Visualization Code Synthesis via Multi-Path Reasoning and Feedback-Driven Optimization [13.178750787401263]
  VisPath handles underspecified queries through structured, multi-stage processing.<n>It begins by reformulating the user input via Chain-of-Thought prompting.<n> VisPath generates targeted feedback that is aggregated to synthesize an optimal final result.
  arXiv  Detail & Related papers  (2025-02-16T14:09:42Z)
• HMGIE: Hierarchical and Multi-Grained Inconsistency Evaluation for Vision-Language Data Cleansing [54.970275599061594]
  We design an adaptive evaluation framework, called Hierarchical and Multi-Grained Inconsistency Evaluation (HMGIE)<n>HMGIE can provide multi-grained evaluations covering both accuracy and completeness for various image-caption pairs.<n>To verify the efficacy and flexibility of the proposed framework, we construct MVTID, an image-caption dataset with diverse types and granularities of inconsistencies.
  arXiv  Detail & Related papers  (2024-12-07T15:47:49Z)
• MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs [61.56904387052982]
  This paper proposes a new visual grounding task called multi-context visual grounding.<n>It aims to localize instances of interest across multiple images based on open-ended text prompts.<n>We benchmark over 20 state-of-the-art MLLMs and foundation models with potential multi-context visual grounding capabilities.
  arXiv  Detail & Related papers  (2024-10-16T07:52:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.