ViGoR-Bench: How Far Are Visual Generative Models From Zero-Shot Visual Reasoners?

• URL: http://arxiv.org/abs/2603.25823v1
• Date: Thu, 26 Mar 2026 18:40:09 GMT
• Title: ViGoR-Bench: How Far Are Visual Generative Models From Zero-Shot Visual Reasoners?
• Abstract summary: ViGoR Vision-Gnerative Reasoning-centric Benchmark is a unified framework designed to dismantle this mirage.<n>ViGoR distinguishes itself through four key innovations: 1) holistic cross-modal coverage bridging Image-to-Image and Video tasks; 2) a dual-track mechanism evaluating both intermediate processes and final results; 3) an evidence-grounded automated judge ensuring high human alignment.<n> Experiments on over 20 leading models reveal that even state-of-the-art systems harbor significant reasoning deficits.
• Score: 48.43592287326448
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Beneath the stunning visual fidelity of modern AIGC models lies a "logical desert", where systems fail tasks that require physical, causal, or complex spatial reasoning. Current evaluations largely rely on superficial metrics or fragmented benchmarks, creating a ``performance mirage'' that overlooks the generative process. To address this, we introduce ViGoR Vision-G}nerative Reasoning-centric Benchmark), a unified framework designed to dismantle this mirage. ViGoR distinguishes itself through four key innovations: 1) holistic cross-modal coverage bridging Image-to-Image and Video tasks; 2) a dual-track mechanism evaluating both intermediate processes and final results; 3) an evidence-grounded automated judge ensuring high human alignment; and 4) granular diagnostic analysis that decomposes performance into fine-grained cognitive dimensions. Experiments on over 20 leading models reveal that even state-of-the-art systems harbor significant reasoning deficits, establishing ViGoR as a critical ``stress test'' for the next generation of intelligent vision models. The demo have been available at https://vincenthancoder.github.io/ViGoR-Bench/

Related papers

• VGI-Bench: Probing Visual Intelligence in Video Generation Models [69.37282368476755]
  VGI-bench is a fine-grained evaluation of visual reasoning capabilities of video generation models.<n>Our evaluations show that current generative systems can solve a subset of visually grounded reasoning tasks, but remain far from reliable.<n>We hope VGI-bench will help stimulate the development of next-generation video generation models.
  arXiv  Detail & Related papers  (2026-08-20T02:56:56Z)
• PokeGym: A Visually-Driven Long-Horizon Benchmark for Vision-Language Models [33.89873575295867]
  We introduce PokeGym, a visually-driven benchmark instantiated within Pokemon Legends: Z-A, a visually complex 3D open-world Role-Playing Game.<n> PokeGym enforces strict code-level isolation: agents operate solely on raw RGB observations while an independent evaluator verifies success via memory scanning.<n>The benchmark comprises 30 tasks (30-220 steps) spanning navigation, interaction, and mixed scenarios, with three instruction granularities (Visual-Guided, Step-Guided, Goal-Only) to systematically deconstruct visual grounding, semantic reasoning, and autonomous exploration capabilities.
  arXiv  Detail & Related papers  (2026-04-09T15:12:36Z)
• ViThinker: Active Vision-Language Reasoning via Dynamic Perceptual Querying [15.728211622542267]
  ViThinker is a framework that enables vision-language models to autonomously generate decision tokens triggering the synthesis of expert-aligned visual features on demand.<n>ViThinker internalizes vision-expert capabilities during training, performing generative mental simulation during inference without external tool calls.
  arXiv  Detail & Related papers  (2026-02-02T22:29:57Z)
• VIPER: Process-aware Evaluation for Generative Video Reasoning [64.86465792516658]
  We introduce VIPER, a comprehensive benchmark spanning 16 tasks across temporal, structural, symbolic, spatial, physics, and planning reasoning.<n>Our experiments reveal that state-of-the-art video models achieve only about 20% POC@1.0 and exhibit a significant outcome-hacking.
  arXiv  Detail & Related papers  (2025-12-31T16:31:59Z)
• MMGR: Multi-Modal Generative Reasoning [97.44203203196481]
  We introduce MMGR, a principled evaluation framework based on five reasoning abilities.<n> MMGR evaluates generative reasoning across three domains: Abstract Reasoning, Embodied Navigation, and Physical Commonsense.<n>We benchmark leading video models (Veo-3, Sora-2, Wan-2.2) and image models (Nano-banana, Nano-banana Pro, GPT-4o-image, Qwen-image)
  arXiv  Detail & Related papers  (2025-12-16T18:58:04Z)
• ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction [35.24704057622881]
  Embodied cognition argues that intelligence arises from sensorimotor interaction rather than passive observation.<n>We introduce ENACT, a benchmark that casts evaluation of embodied cognition as world modeling from egocentric interaction.
  arXiv  Detail & Related papers  (2025-11-26T00:06:02Z)
• Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark [48.02995109011304]
  Video generation models have emerged as potential world simulators through Chain-of-Frames (CoF) reasoning.<n>Existing benchmarks, focusing on fidelity or alignment, do not assess CoF reasoning.<n>We introduce Gen-ViRe, a framework grounded in cognitive science and real-world AI applications.
  arXiv  Detail & Related papers  (2025-11-17T19:11:39Z)
• BLINK-Twice: You see, but do you observe? A Reasoning Benchmark on Visual Perception [67.89135437537179]
  We introduce BLINK-Twice, a vision-centric reasoning benchmark grounded in challenging perceptual tasks.<n>Instead of relying on external knowledge, our tasks require models to reason from visual content alone.<n>Compared to prior perception benchmarks, it moves beyond shallow perception and requires fine-grained observation and analytical reasoning.
  arXiv  Detail & Related papers  (2025-10-10T13:14:13Z)
• VLMs have Tunnel Vision: Evaluating Nonlocal Visual Reasoning in Leading VLMs [18.349695067647012]
  Visual Language Models excel at complex visual tasks such as VQA and chart understanding, yet recent work suggests they struggle with simple tests.<n>We present an evaluation that tests vision-language models' capacity for nonlocal visual reasoning.<n>Our findings show that despite gains in raw visual acuity, current models lack core visual reasoning capabilities.
  arXiv  Detail & Related papers  (2025-07-04T23:15:52Z)
• Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models [51.900488744931785]
  We introduce the Visual Graph Arena (VGA) to evaluate and improve AI systems' capacity for visual abstraction.<n>Humans achieve near-perfect accuracy across tasks, while models totally failed on isomorphism detection and showed limited success in path/cycle tasks.<n>By isolating the challenge of representation-invariant reasoning, the VGA provides a framework to drive progress toward human-like conceptualization in AI visual models.
  arXiv  Detail & Related papers  (2025-06-06T17:06:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.