Image Generators are Generalist Vision Learners

• URL: http://arxiv.org/abs/2604.20329v1
• Date: Wed, 22 Apr 2026 08:23:48 GMT
• Title: Image Generators are Generalist Vision Learners
• Abstract summary: We show that image generation training serves a role similar to language understanding and reasoning pretraining.<n>We introduce Vision Banana, a generalist model built by instruction-tuning Nano Banana Pro (NBP) on a mixture of its original training data and vision task data.<n>Our model achieves SOTA results on a variety of vision tasks involving both 2D and 3D understanding, beating or rivaling zero-shot domain-specialists.
• Score: 71.11980587450198
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining. While it has long been conjectured that the ability to create visual content implies an ability to understand it, there has been limited evidence that generative vision models have developed strong understanding capabilities. In this work, we demonstrate that image generation training serves a role similar to LLM pretraining, and lets models learn powerful and general visual representations that enable SOTA performance on various vision tasks. We introduce Vision Banana, a generalist model built by instruction-tuning Nano Banana Pro (NBP) on a mixture of its original training data alongside a small amount of vision task data. By parameterizing the output space of vision tasks as RGB images, we seamlessly reframe perception as image generation. Our generalist model, Vision Banana, achieves SOTA results on a variety of vision tasks involving both 2D and 3D understanding, beating or rivaling zero-shot domain-specialists, including Segment Anything Model 3 on segmentation tasks, and the Depth Anything series on metric depth estimation. We show that these results can be achieved with lightweight instruction-tuning without sacrificing the base model's image generation capabilities. The superior results suggest that image generation pretraining is a generalist vision learner. It also shows that image generation serves as a unified and universal interface for vision tasks, similar to text generation's role in language understanding and reasoning. We could be witnessing a major paradigm shift for computer vision, where generative vision pretraining takes a central role in building Foundational Vision Models for both generation and understanding.

Related papers

• Video Generation Models are General-Purpose Vision Learners [62.28137589984863]
  We argue that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision.<n>We introduce GenCeption, which leverages a pre-trained video video diffusion backbone to define a feed-forward perception model.<n> Empirical results demonstrate that GenCeption achieves state-of-the-art performance across a diverse suite of tasks.
  arXiv  Detail & Related papers  (2026-07-10T01:09:06Z)
• Unified Pix Token And Word Token Generative Language Model [1.5469452301122175]
  We propose a new model to unify pix token and word token into the generative language model.<n>New model also features with each pix of image having its own token embedding, color folding, global conditional attention approximation and image unsupervised pretraining.
  arXiv  Detail & Related papers  (2026-05-13T18:38:51Z)
• UniHetero: Could Generation Enhance Understanding for Vision-Language-Model at Large Data Scale? [17.49769409149882]
  A common assumption in unified vision-language models is that adding generation will naturally strengthen understanding.<n>At 200M+ pretraining samples, generation helps understanding only when it operates at the semantic level.<n>Unified generation-understanding demonstrates a superior scaling trend compared to understanding alone.
  arXiv  Detail & Related papers  (2025-12-29T14:49:50Z)
• Visual Jigsaw Post-Training Improves MLLMs [58.29961336087896]
  We introduce Visual Jigsaw, a generic self-supervised post-training framework designed to strengthen visual understanding in large language models (MLLMs)<n>Visual Jigsaw is formulated as a general ordering task: visual inputs are partitioned, shuffled, and the model must reconstruct the visual information by producing the correct permutation in natural language.<n>Extensive experiments demonstrate substantial improvements in fine-grained perception, temporal reasoning, and 3D spatial understanding.
  arXiv  Detail & Related papers  (2025-09-29T17:59:57Z)
• From Generation to Generalization: Emergent Few-Shot Learning in Video Diffusion Models [65.0487600936788]
  Video Diffusion Models (VDMs) have emerged as powerful generative tools capable of synthesizing high-quality content.<n>We argue that VDMs naturally push to probe structured representations and an implicit understanding of the visual world.<n>Our method transforms each task into a visual transition, enabling the training of LoRA weights on short input- sequences.
  arXiv  Detail & Related papers  (2025-06-08T20:52:34Z)
• Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning [92.57052246970254]
  We propose to enable the collaborative co-evolution of visual comprehension and generation.<n>We introduce a two-stage training approach: supervised fine-tuning teaches the MLLM with the foundational ability to generate genuine CoT.<n>We unlock the Aha moment in visual generation, advancing MLLMs from text-to-image tasks to unified image generation.
  arXiv  Detail & Related papers  (2025-06-02T09:39:28Z)
• Heuristic Vision Pre-Training with Self-Supervised and Supervised Multi-Task Learning [0.0]
  We propose a novel pre-training framework by adopting both self-supervised and supervised visual pre-text tasks in a multi-task manner. Results show that our pre-trained models can deliver results on par with or better than state-of-the-art (SOTA) results on multiple visual tasks.
  arXiv  Detail & Related papers  (2023-10-11T14:06:04Z)
• DeViL: Decoding Vision features into Language [53.88202366696955]
  Post-hoc explanation methods have often been criticised for abstracting away the decision-making process of deep neural networks. In this work, we would like to provide natural language descriptions for what different layers of a vision backbone have learned. We train a transformer network to translate individual image features of any vision layer into a prompt that a separate off-the-shelf language model decodes into natural language.
  arXiv  Detail & Related papers  (2023-09-04T13:59:55Z)
• GPT4Image: Large Pre-trained Models Help Vision Models Learn Better on Perception Task [47.1857510710807]
  We present a new learning framework, dubbed GPT4Image, where the knowledge of the large pre-trained models are extracted to help CNNs and ViTs learn better representations.<n>We conduct extensive experiments to verify the effectiveness of the proposed algorithm on various visual perception tasks.
  arXiv  Detail & Related papers  (2023-06-01T14:02:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.