Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models

• URL: http://arxiv.org/abs/2605.08841v1
• Date: Sat, 09 May 2026 09:52:23 GMT
• Title: Illusion-Aware Visual Preprocessing and Anti-Illusion Prompting for Classic Illusion Understanding in Vision-Language Models
• Abstract summary: Vision-Language Models (VLMs) exhibit systematic bias toward visual illusions, recalling memorized facts rather than perceiving actual visual differences.<n>This paper presents a training-free framework for the 5th DataCV Challenge Task 1 at CVPR 2026, addressing this perception-versus-memory conflict.<n>Our solution secured 2nd place in the challenge, only 0.47% behind the 1st-place solution.
• Score: 11.342804429826936
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vision-Language Models (VLMs) exhibit systematic bias toward visual illusions, recalling memorized facts rather than perceiving actual visual differences. This paper presents a training-free framework for the 5th DataCV Challenge Task 1 at CVPR 2026, addressing this perception-versus-memory conflict through three complementary strategies:(1) illusion-aware image preprocessing that weakens illusion-inducing context via type-specific transformations (edge extraction, color isolation, morphological processing, and reference-line overlay), (2) anti-illusion prompt engineering guiding VLMs toward qualitative visual comparison, and (3) multi-vote ensemble that further improves robustness. Our method achieves 90.48% accuracy on the official 630-image test set using Claude (claude-opus-4-6) with 5-vote majority ensemble, and 98.41% on a human-verified subset. The approach requires no finetuning, relying solely on visual manipulation and prompt design. Our solution secured 2nd place in the challenge, only 0.47% behind the 1st-place solution. Code is available at https://github.com/jasminezz/sf-illusion-aware-vlm.git.

Related papers

• Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective [76.02772556944073]
  Environmental illusions are naturally existing yet overlooked phenomena in real-world driving environments.<n>They can disturb visual perception, leading to misinterpretation of the scene and posing serious safety risks to autonomous driving systems.<n>We introduce the first benchmark, LanEvil++, for evaluating the robustness of lane perception under environmental illusions.
  arXiv  Detail & Related papers  (2026-07-07T03:15:00Z)
• Seeing the Evidence, Missing the Answer: Tool-Guided Vision-Language Models on Visual Illusions [2.9521041993295607]
  Vision-language models (VLMs) exhibit a systematic bias when confronted with classic optical illusions.<n>We present a tool-guided inference framework that addresses this failure mode without any model training.<n>We report three empirical observations that we believe warrant additional investigation.
  arXiv  Detail & Related papers  (2026-03-31T08:35:10Z)
• Do VLMs Perceive or Recall? Probing Visual Perception vs. Memory with Classic Visual Illusions [38.42074629749954]
  Large Vision-Language Models (VLMs) often answer classic visual illusions "correctly" on original images, yet persist with the same responses when illusion factors are inverted.<n>This raises a fundamental question: do VLMs perceive visual changes or merely recall memorized patterns?<n>This paper introduces VI-Probe, a controllable visual-illusion framework with graded perturbations and matched visual controls.
  arXiv  Detail & Related papers  (2026-01-29T18:59:24Z)
• Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology [87.65242416688146]
  TreeBench is a diagnostic benchmark for visual grounded reasoning.<n>TreeVGR is a training paradigm to supervise localization and reasoning jointly with reinforcement learning.
  arXiv  Detail & Related papers  (2025-07-10T17:59:58Z)
• ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs [98.27348724529257]
  We introduce ViCrit (Visual Caption Hallucination Critic), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions.<n>Models trained with the ViCrit Task exhibit substantial gains across a variety of vision-language models benchmarks.
  arXiv  Detail & Related papers  (2025-06-11T19:16:54Z)
• SemVink: Advancing VLMs' Semantic Understanding of Optical Illusions via Visual Global Thinking [31.356290235986332]
  Vision-language models (VLMs) excel in semantic tasks but falter at a core human capability.<n>We introduce HC-Bench, a benchmark of 112 images with hidden text, objects, and illusions.<n>We propose SemVink (Semantic Visual Thinking), which unlocks >99% accuracy by eliminating redundant visual noise.
  arXiv  Detail & Related papers  (2025-06-03T12:33:47Z)
• IllusionBench+: A Large-scale and Comprehensive Benchmark for Visual Illusion Understanding in Vision-Language Models [56.34742191010987]
  Current Visual Language Models (VLMs) show impressive image understanding but struggle with visual illusions.<n>We introduce IllusionBench, a comprehensive visual illusion dataset that encompasses classic cognitive illusions and real-world scene illusions.<n>We design trap illusions that resemble classical patterns but differ in reality, highlighting issues in SOTA models.
  arXiv  Detail & Related papers  (2025-01-01T14:10:25Z)
• Aligning Modalities in Vision Large Language Models via Preference Fine-tuning [67.62925151837675]
  In this work, we frame the hallucination problem as an alignment issue, tackle it with preference tuning. Specifically, we propose POVID to generate feedback data with AI models. We use ground-truth instructions as the preferred response and a two-stage approach to generate dispreferred data. In experiments across broad benchmarks, we show that we can not only reduce hallucinations, but improve model performance across standard benchmarks, outperforming prior approaches.
  arXiv  Detail & Related papers  (2024-02-18T00:56:16Z)
• HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models [69.52245481329899]
  We introduce HallusionBench, a benchmark for the evaluation of image-context reasoning. The benchmark comprises 346 images paired with 1129 questions, all meticulously crafted by human experts. In our evaluation on HallusionBench, we benchmarked 15 different models, highlighting a 31.42% question-pair accuracy achieved by the state-of-the-art GPT-4V.
  arXiv  Detail & Related papers  (2023-10-23T04:49:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.