Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling

• URL: http://arxiv.org/abs/2604.28185v1
• Date: Thu, 30 Apr 2026 17:59:02 GMT
• Title: Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling
• Abstract summary: We argue that the field should move beyond appearance synthesis toward intelligent visual generation.<n>We introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation.<n>We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, and sampling acceleration.
• Score: 183.5907213030813
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent visual generation models have made major progress in photorealism, typography, instruction following, and interactive editing, yet they still struggle with spatial reasoning, persistent state, long-horizon consistency, and causal understanding. We argue that the field should move beyond appearance synthesis toward intelligent visual generation: plausible visuals grounded in structure, dynamics, domain knowledge, and causal relations. To frame this shift, we introduce a five-level taxonomy: Atomic Generation, Conditional Generation, In-Context Generation, Agentic Generation, and World-Modeling Generation, progressing from passive renderers to interactive, agentic, world-aware generators. We analyze key technical drivers, including flow matching, unified understanding-and-generation models, improved visual representations, post-training, reward modeling, data curation, synthetic data distillation, and sampling acceleration. We further show that current evaluations often overestimate progress by emphasizing perceptual quality while missing structural, temporal, and causal failures. By combining benchmark review, in-the-wild stress tests, and expert-constrained case studies, this roadmap offers a capability-centered lens for understanding, evaluating, and advancing the next generation of intelligent visual generation systems.

Related papers

• Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models [34.560908829375585]
  We propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step.<n>Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts.<n>We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency.<n>This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.
  arXiv  Detail & Related papers  (2026-05-15T09:48:46Z)
• DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving [49.11389494068169]
  We present DrivingGen, the first comprehensive benchmark for generative driving world models.<n>DrivingGen combines a diverse evaluation dataset curated from both driving datasets and internet-scale video sources.<n>General models look better but break physics, while driving-specific ones capture motion realistically but lag in visual quality.
  arXiv  Detail & Related papers  (2026-01-04T13:36:21Z)
• Simulating the Visual World with Artificial Intelligence: A Roadmap [48.64639618440864]
  Video generation is shifting from generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.<n>This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components.<n>We trace the progression of video generation through four generations, culminating in a video generation model that embodies intrinsic physical plausibility.
  arXiv  Detail & Related papers  (2025-11-11T18:59:50Z)
• Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges [55.41538606569424]
  Review focuses on four key domains: image generation, text generation, molecular profile-morphology generation, and other specialized generation applications.<n>We trace the evolution of content generation architectures -- from early generative adversarial networks to recent advances in diffusion models and generative vision-language models.<n>Review concludes with a discussion of open challenges and prospective research directions, with an emphasis on developing integrated and clinically deployable generation systems.
  arXiv  Detail & Related papers  (2025-05-16T08:44:50Z)
• RealRAG: Retrieval-augmented Realistic Image Generation via Self-reflective Contrastive Learning [54.07026389388881]
  We present the first real-object-based retrieval-augmented generation framework (RealRAG)<n>RealRAG augments fine-grained and unseen novel object generation by learning and retrieving real-world images to overcome the knowledge gaps of generative models.<n>Our framework integrates fine-grained visual knowledge for the generative models, tackling the distortion problem and improving the realism for fine-grained object generation.
  arXiv  Detail & Related papers  (2025-02-02T16:41:54Z)
• Generative Physical AI in Vision: A Survey [78.07014292304373]
  Gene Artificial Intelligence (AI) has rapidly advanced the field of computer vision by enabling machines to create and interpret visual data with unprecedented sophistication.<n>This transformation builds upon a foundation of generative models to produce realistic images, videos, and 3D/4D content.<n>As generative models evolve to increasingly integrate physical realism and dynamic simulation, their potential to function as "world simulators" expands.
  arXiv  Detail & Related papers  (2025-01-19T03:19:47Z)
• Interactive Visual Assessment for Text-to-Image Generation Models [28.526897072724662]
  We propose DyEval, a dynamic interactive visual assessment framework for generative models. DyEval features an intuitive visual interface that enables users to interactively explore and analyze model behaviors. Our framework provides valuable insights for improving generative models and has broad implications for advancing the reliability and capabilities of visual generation systems.
  arXiv  Detail & Related papers  (2024-11-23T10:06:18Z)
• A Survey on Vision Autoregressive Model [15.042485771127346]
  Autoregressive models have demonstrated impressive performance in natural language processing (NLP) Inspired by their notable success in NLP field, autoregressive models have been intensively investigated recently for computer vision. This paper provides a systematic review of vision autoregressive models, including the development of a taxonomy of existing methods.
  arXiv  Detail & Related papers  (2024-11-13T14:59:41Z)
• KITTEN: A Knowledge-Intensive Evaluation of Image Generation on Visual Entities [93.74881034001312]
  KITTEN is a benchmark for Knowledge-InTensive image generaTion on real-world ENtities.<n>We conduct a systematic study of the latest text-to-image models and retrieval-augmented models.<n>Analysis shows that even advanced text-to-image models fail to generate accurate visual details of entities.
  arXiv  Detail & Related papers  (2024-10-15T17:50:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.