MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts

• URL: http://arxiv.org/abs/2607.18673v2
• Date: Wed, 22 Jul 2026 07:26:05 GMT
• Title: MissingBench-Verified: Probing Vision-Language Models' Inability to Detect Missing Object Parts
• Abstract summary: We present MissingBench-Verified, a benchmark designed to evaluate a specific and practically relevant scenario.<n>We observe consistent and significant failure rates that persist even when external tool evidence explicitly contradicts the model's visual perception.<n>We find that existing mitigation strategies, including tool-assisted verification, autonomous visual reasoning, longer reasoning durations, and fine-tuning on an easier dataset, provide negligible improvement.
• Score: 10.608824155356377
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision Language Models (VLMs) are well known for hallucinating non-existent objects in images. Objects with missing parts present a unique challenge for VLMs, stemming from both real-world knowledge bias and the scarcity of such images in training data. We present MissingBench-Verified, a benchmark designed to evaluate a specific and practically relevant scenario: when vision-language models fail to recognize that an essential component of an object has been removed. Across ten leading models, we observe consistent and significant failure rates that persist even when external tool evidence explicitly contradicts the model's visual perception. We further ask whether granting models access to image processing tools (e.g., cropping, contrast adjustment) enables autonomous inspection to resolve these failures. We find that existing mitigation strategies, including tool-assisted verification, autonomous visual reasoning, longer reasoning durations, and fine-tuning on an easier dataset, provide negligible improvement, indicating that this failure mode cannot be addressed through current prompting or post-hoc correction techniques. Our findings highlight a fundamental limitation of current VLM for inspection and monitoring tasks and underscore the need for architectural or training-level interventions that enable models to override internal expectations when confronted with contradictory evidence.

Related papers

• Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs [35.17881745040865]
  Tool-augmented multimodal reasoning integrates external tools into multimodal large language models.<n>We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning.
  arXiv  Detail & Related papers  (2026-08-29T17:22:08Z)
• Can Vision-Language Models Reason about AI Edits in Images? [78.04125956307838]
  Vision-Language Models (VLMs) offer a promising alternative due to their strong visual understanding and reasoning capabilities.<n>We propose a GRPO-based training framework that utilizes simple accuracy and format rewards.<n>Our approach achieves competitive detection and localization performance compared to state-of-the-art image forgery detectors.
  arXiv  Detail & Related papers  (2026-07-30T16:23:40Z)
• Does It Fail to See or Fail to Know? Attributing Errors in Vision-Language Models [33.564453692157095]
  Uncertainty should not only indicate whether the model is likely to fail but also diagnose why it is uncertain.<n>We propose a unified framework for disentangling failure modes and investigate whether pre-generation signals can predict these failure sources.<n>Our main finding is that recognition-related failures are best captured by visual-token representations, while failures that remain after recognition are better captured by prompt-conditioned hidden states.
  arXiv  Detail & Related papers  (2026-07-06T05:11:45Z)
• Advancing Creative Physical Intelligence in Large Multimodal Models [62.56522271769017]
  We introduce MM-CreativityBench, a benchmark for affordance-grounded creative tool use in visually rich, physically constrained environments.<n>Our experiments show that current LMMs often fall short, not due to lack of generative capability, but because they do not sustain grounded exploration.<n>Motivated by this failure mode, we propose affordance-grounded alignment, which casts creative tool use as a preference learning problem.
  arXiv  Detail & Related papers  (2026-05-25T23:59:02Z)
• ReViP: Reducing False Completion in Vision-Language-Action Models with Vision-Proprioception Rebalance [50.05984919728878]
  We present ReViP, a novel VLA framework with Vision-Proprioception Rebalance to enhance visual grounding and robustness under perturbations.<n>Specifically, we use an external VLM as a task-stage observer to extract real-time task-centric visual cues from visual observations.<n>To evaluate false completion, we propose the first False-Completion Benchmark Suite built on LIBERO with controlled settings such as Object-Drop.
  arXiv  Detail & Related papers  (2026-01-23T11:31:07Z)
• From Sight to Insight: Unleashing Eye-Tracking in Weakly Supervised Video Salient Object Detection [60.11169426478452]
  This paper aims to introduce fixation information to assist the detection of salient objects under weak supervision.<n>We propose a Position and Semantic Embedding (PSE) module to provide location and semantic guidance during the feature learning process.<n>An Intra-Inter Mixed Contrastive (MCII) model improves thetemporal modeling capabilities under weak supervision.
  arXiv  Detail & Related papers  (2025-06-30T05:01:40Z)
• Are vision language models robust to uncertain inputs? [5.249651874118556]
  We show that newer and larger vision language models exhibit improved robustness compared to earlier models, but still suffer from a tendency to strictly follow instructions.<n>For natural images such as ImageNet, this limitation can be overcome without pipeline modifications.<n>We propose a novel mechanism based on caption diversity to reveal a model's internal uncertainty.
  arXiv  Detail & Related papers  (2025-05-17T03:16:49Z)
• Mitigating Low-Level Visual Hallucinations Requires Self-Awareness: Database, Model and Training Strategy [53.07517728420411]
  We introduce the first instruction database specifically focused on hallucinations in low-level vision tasks.<n>We propose the Self-Awareness Failure Elimination (SAFEQA) model to improve the perception and comprehension abilities of the model in low-level vision tasks.<n>We conduct comprehensive experiments on low-level vision tasks, with the results demonstrating that our proposed method significantly enhances self-awareness of the model in these tasks and reduces hallucinations.
  arXiv  Detail & Related papers  (2025-03-26T16:05:01Z)
• Unsupervised Model Diagnosis [49.36194740479798]
  This paper proposes Unsupervised Model Diagnosis (UMO) to produce semantic counterfactual explanations without any user guidance. Our approach identifies and visualizes changes in semantics, and then matches these changes to attributes from wide-ranging text sources.
  arXiv  Detail & Related papers  (2024-10-08T17:59:03Z)
• Evaluation and Comparison of Visual Language Models for Transportation Engineering Problems [16.49637074299509]
  We have explored state-of-the-art vision language models (VLM) for vision-based transportation engineering tasks. The image classification task involves congestion detection and crack identification, whereas, for object detection, helmet violations were identified. We have applied open-source models such as CLIP, BLIP, OWL-ViT, Llava-Next, and closed-source GPT-4o to evaluate the performance of these VLM models.
  arXiv  Detail & Related papers  (2024-09-03T20:24:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.