Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation

• URL: http://arxiv.org/abs/2609.07884v1
• Date: Mon, 07 Sep 2026 18:48:57 GMT
• Title: Are Image Generators Zero-Shot Perceivers? A Rigorous Evaluation
• Abstract summary: We ask how far image generators can go on public visual perception benchmarks in a zero-shot setting.<n>We introduce ProbeGen, a benchmark for zero-shot generative perception that casts monocular depth estimation.<n>We observe that pretrained image generators show measurable zero-shot perceptual competence, but with a clear trade-off.
• Score: 54.18771863630389
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work, such as Vision Banana, shows that lightweight instruction tuning can enable an image generator to achieve state-of-the-art performance across multiple visual perception tasks. Motivated by this perspective, we ask how far image generators can go on public visual perception benchmarks in a zero-shot setting. We introduce ProbeGen, a benchmark for zero-shot generative perception that casts monocular depth estimation, referring/reasoning segmentation, and object counting as conditional generation tasks specified through text prompts, and compares 20 models in total---including proprietary and open-weight image generators, specialist perception models, and MLLMs---across 11 published benchmarks. We observe that pretrained image generators show measurable zero-shot perceptual competence, but with a clear trade-off: specialist models remain stronger for in-distribution accuracy and efficiency, while generative models are often more robust under distribution shift and better at compositional semantic reasoning. We hope this study helps establish zero-shot generative perception as a meaningful research direction and provides a useful foundation for future work at the intersection of visual generation and understanding.

Related papers

• SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders [61.07733730455141]
  We investigate how much information about image authenticity is already encoded in modern multimodal vision representations.<n>We develop a representation-aware data curation strategy that selects a compact set of representative generators for training.<n>Experiments show that combining frozen multimodal representations with carefully curated training data provides a simple and effective approach to AI-generated image detection.
  arXiv  Detail & Related papers  (2026-06-07T13:56:17Z)
• RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflection [18.52946282633359]
  RL-RIG is a Reinforcement Learning framework for Reflection-based Image Generation.<n>We develop Reflection-GRPO to train the VLM Actor for edit prompts and the Image Editor for better image quality under a given prompt.<n> Experimental results show that RL-RIG outperforms existing state-of-the-art open-source models by up to 11% in terms of controllable and precise spatial reasoning in image generation.
  arXiv  Detail & Related papers  (2026-02-23T15:39:53Z)
• MiraGe: Multimodal Discriminative Representation Learning for Generalizable AI-Generated Image Detection [32.662682253295486]
  We propose Multimodal Discriminative Learning for Generalizable AI-generated Image Detection (MiraGegenerator)<n>We apply multimodal prompt learning to further refine these principles into CLIP, leveraging text embeddings as semantic anchors for effective discriminative representation learning.<n>MiraGegenerator achieves state-of-the-art performance, maintaining robustness even against unseen generators like Sora.
  arXiv  Detail & Related papers  (2025-08-03T00:19:18Z)
• Prefilled responses enhance zero-shot detection of AI-generated images [2.6581858762749997]
  We explore pre-trained Vision-Language Models (VLMs) for zero-shot detection of AI-generated images.<n>We evaluate VLM performance on three benchmarks encompassing synthetic images of human faces, objects, and animals.<n>In particular, prefilling a VLM response with the task-aligned phrase "Let's examine the style and the synthesis artifacts" improves the Macro F1 scores of three widely used open-source VLMs by up to 24%.
  arXiv  Detail & Related papers  (2025-05-20T22:44:04Z)
• Manifold Induced Biases for Zero-shot and Few-shot Detection of Generated Images [4.597145995119321]
  We identify two main gaps: (1) a lack of theoretical grounding for the methods, and (2) significant room for performance improvements in zero-shot and few-shot regimes.<n>Specifically, we explore the biases of the implicit probability manifold, captured by a pre-trained diffusion model.<n>Through score-function analysis, we approximate the curvature, gradient, and bias towards points on the probability manifold, establishing criteria for detection in the zero-shot regime.<n> Empirical results across 20 generative models demonstrate that our method outperforms current approaches in both zero-shot and few-shot settings.
  arXiv  Detail & Related papers  (2025-04-21T22:39:24Z)
• GH-Feat: Learning Versatile Generative Hierarchical Features from GANs [61.208757845344074]
  We show that a generative feature learned from image synthesis exhibits great potentials in solving a wide range of computer vision tasks. We first train an encoder by considering the pretrained StyleGAN generator as a learned loss function. The visual features produced by our encoder, termed as Generative Hierarchical Features (GH-Feat), highly align with the layer-wise GAN representations.
  arXiv  Detail & Related papers  (2023-01-12T21:59:46Z)
• Composing Ensembles of Pre-trained Models via Iterative Consensus [95.10641301155232]
  We propose a unified framework for composing ensembles of different pre-trained models. We use pre-trained models as "generators" or "scorers" and compose them via closed-loop iterative consensus optimization. We demonstrate that consensus achieved by an ensemble of scorers outperforms the feedback of a single scorer.
  arXiv  Detail & Related papers  (2022-10-20T18:46:31Z)
• GRIT: General Robust Image Task Benchmark [32.556726698322755]
  We introduce the General Robust Image Task (GRIT) benchmark. GRIT evaluates the performance, robustness, and calibration of a vision system across a variety of image prediction tasks, concepts, and data sources. By providing a unified platform for thorough assessment of skills and concepts learned by a vision model, we hope GRIT catalyzes the development of performant and robust general purpose vision systems.
  arXiv  Detail & Related papers  (2022-04-28T17:13:23Z)
• Learning to Prompt for Vision-Language Models [82.25005817904027]
  Vision-language pre-training has emerged as a promising alternative for representation learning. It shifts from the tradition of using images and discrete labels for learning a fixed set of weights, seen as visual concepts, to aligning images and raw text for two separate encoders. Such a paradigm benefits from a broader source of supervision and allows zero-shot transfer to downstream tasks.
  arXiv  Detail & Related papers  (2021-09-02T17:57:31Z)
• Generative Hierarchical Features from Synthesizing Images [65.66756821069124]
  We show that learning to synthesize images can bring remarkable hierarchical visual features that are generalizable across a wide range of applications. The visual feature produced by our encoder, termed as Generative Hierarchical Feature (GH-Feat), has strong transferability to both generative and discriminative tasks.
  arXiv  Detail & Related papers  (2020-07-20T18:04:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.