From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation

• URL: http://arxiv.org/abs/2608.18076v2
• Date: Tue, 25 Aug 2026 10:09:09 GMT
• Title: From Corpora to Co-Evolving Capabilities: Capability-Centric Data Design for Generalist Image Generation
• Abstract summary: Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation.<n>We present a textbfcapability-driven data infrastructure that couples capability-specific supervision construction with capability-aligned curriculum scheduling.<n>At scale, the framework curates a 440M-image T2I corpus, 120M editing pairs, and over 27M image-entity pairs.
• Score: 22.386044581413316
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale image generation has benefited from advances in data scale, quality, rebalancing, and recaptioning, yet conventional pipelines typically optimize task-specific datasets in isolation. A central challenge is not only how to curate each task-specific corpus, but also how to organize heterogeneous supervision according to the dependencies among generative capabilities. We present a \textbf{capability-driven data infrastructure} that couples capability-specific supervision construction with capability-aligned curriculum scheduling. Its three specialized yet interoperable data engines build complementary relational supervision for text-image grounding, inter-image transformation, and image-knowledge association, while caption experts align T2I and editing supervision across tasks and granularities. A multi-stage curriculum jointly evolves task composition, visual-concept distribution, data quality, and image resolution along the dependency order of capability acquisition, with capability-aware evaluation closing the loop through targeted retrieval, expert construction, and gap-aware resampling. At scale, the framework curates a 440M-image T2I corpus, 120M editing pairs, and over 27M image-entity pairs. With this infrastructure, we train multimodal diffusion models at two scales from scratch, with 3B and 6B sizes respectively. We conduct quantitative evaluation on CPI-Bench, along with qualitative evaluations across diverse text-to-image and editing scenarios. Experimental results present broad visual coverage, versatile rendering, and effective transfer across generative capabilities.

Related papers

• CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences [51.86576932800981]
  We introduce CV-Arena, an open benchmark designed to evaluate this capability at professional scales.<n>CV-Arena contains 12K high-resolution real-image instruction pairs spanning 16 instruction-based visual task types.<n>We develop CV-Agent, a lightweight agentic model that combines planning, editing, and verification.
  arXiv  Detail & Related papers  (2026-05-30T23:37:55Z)
• EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding [21.73510227758125]
  EchoGen is a unified framework for layout-to-image generation and image grounding.<n>Image grounding possesses strong text and layout understanding abilities, which can compensate for the limitations in layout-to-image generation.<n>Images generated from layouts exhibit high diversity in content, thereby enhancing the robustness of image grounding.
  arXiv  Detail & Related papers  (2026-03-18T17:59:03Z)
• Transferable Optimization Network for Cross-Domain Image Reconstruction [1.4929298667651638]
  We develop a novel transfer learning framework to tackle the challenge of limited training data in image reconstruction problems.<n>In the first step, we train a powerful universal feature-extractor that is capable of learning important knowledge from large, heterogeneous data sets.<n>In the second step, we train a task-specific domain-adapter for a new target domain or task with only a limited amount of data available for training.
  arXiv  Detail & Related papers  (2026-03-08T22:20:03Z)
• Agentic Planning with Reasoning for Image Styling via Offline RL [66.10749901925941]
  Direct prompt-based editing fails on complex transformations because vague and subjective prompts often require nuanced understanding of what should be changed in the image.<n>We present a tool-based agentic RL post-training framework that addresses this through structured planning with chain-of-thought reasoning.
  arXiv  Detail & Related papers  (2026-03-07T11:14:37Z)
• Factuality Matters: When Image Generation and Editing Meet Structured Visuals [46.627460447235855]
  We construct a large-scale dataset of 1.3 million high-quality structured image pairs.<n>We train a unified model that integrates a VLM with FLUX.1 Kontext.<n>A three-stage training curriculum enables progressive feature alignment, knowledge infusion, and reasoning-augmented generation.
  arXiv  Detail & Related papers  (2025-10-06T17:56:55Z)
• DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Data [67.99373622902827]
  DIPO is a framework for controllable generation of articulated 3D objects from a pair of images.<n>We propose a dual-image diffusion model that captures relationships between the image pair to generate part layouts and joint parameters.<n>We propose PM-X, a large-scale dataset of complex articulated 3D objects, accompanied by rendered images, URDF annotations, and textual descriptions.
  arXiv  Detail & Related papers  (2025-05-26T18:55:14Z)
• UniVG: A Generalist Diffusion Model for Unified Image Generation and Editing [59.590505989071175]
  Text-to-Image (T2I) diffusion models have shown impressive results in generating visually compelling images following user prompts.<n>We introduce UniVG, a generalist diffusion model capable of supporting a diverse range of image generation tasks with a single set of weights.
  arXiv  Detail & Related papers  (2025-03-16T21:11:25Z)
• ImageScope: Unifying Language-Guided Image Retrieval via Large Multimodal Model Collective Reasoning [62.61187785810336]
  ImageScope is a training-free, three-stage framework that unifies language-guided image retrieval tasks.<n>In the first stage, we improve the robustness of the framework by synthesizing search intents across varying levels of semantic granularity.<n>In the second and third stages, we reflect on retrieval results by verifying predicate propositions locally, and performing pairwise evaluations globally.
  arXiv  Detail & Related papers  (2025-03-13T08:43:24Z)
• Efficient Concertormer for Image Deblurring and Beyond [87.07963453448328]
  We introduce a novel Concerto Self-Attention (CSA) mechanism designed for image deblurring.<n>By retaining partial information in additional dimensions independent from the self-attention calculations, our method effectively captures global contextual representations with complexity linear to the image size.<n>While our primary objective is single-image motion deblurring, extensive quantitative and qualitative evaluations demonstrate that our approach performs favorably against state-of-the-art methods in other tasks.
  arXiv  Detail & Related papers  (2024-04-09T09:02:21Z)
• RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Model [22.56227565913003]
  We propose a comprehensive remote sensing image building model, termed RSBuilding, developed from the perspective of the foundation model. RSBuilding is designed to enhance cross-scene generalization and task understanding. Our model was trained on a dataset comprising up to 245,000 images and validated on multiple building extraction and change detection datasets.
  arXiv  Detail & Related papers  (2024-03-12T11:51:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.