Conditional Panoramic Image Generation via Masked Autoregressive Modeling

• URL: http://arxiv.org/abs/2505.16862v1
• Date: Thu, 22 May 2025 16:20:12 GMT
• Title: Conditional Panoramic Image Generation via Masked Autoregressive Modeling
• Authors: Chaoyang Wang, Xiangtai Li, Lu Qi, Xiaofan Lin, Jinbin Bai, Qianyu Zhou, Yunhai Tong,
• Abstract summary: We propose a unified framework, Panoramic AutoRegressive model (PAR), which leverages masked autoregressive modeling to address these challenges.<n>To address the inherent discontinuity in existing generative models, we introduce circular padding to enhance spatial coherence.<n>Experiments demonstrate competitive performance in text-to-image generation and panorama outpainting tasks.
• Score: 35.624070746282186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent progress in panoramic image generation has underscored two critical limitations in existing approaches. First, most methods are built upon diffusion models, which are inherently ill-suited for equirectangular projection (ERP) panoramas due to the violation of the identically and independently distributed (i.i.d.) Gaussian noise assumption caused by their spherical mapping. Second, these methods often treat text-conditioned generation (text-to-panorama) and image-conditioned generation (panorama outpainting) as separate tasks, relying on distinct architectures and task-specific data. In this work, we propose a unified framework, Panoramic AutoRegressive model (PAR), which leverages masked autoregressive modeling to address these challenges. PAR avoids the i.i.d. assumption constraint and integrates text and image conditioning into a cohesive architecture, enabling seamless generation across tasks. To address the inherent discontinuity in existing generative models, we introduce circular padding to enhance spatial coherence and propose a consistency alignment strategy to improve generation quality. Extensive experiments demonstrate competitive performance in text-to-image generation and panorama outpainting tasks while showcasing promising scalability and generalization capabilities.

Related papers

• Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model [87.23753533733046]
  We introduce Muddit, a unified discrete diffusion transformer that enables fast and parallel generation across both text and image modalities.<n>Unlike prior unified diffusion models trained from scratch, Muddit integrates strong visual priors from a pretrained text-to-image backbone with a lightweight text decoder.
  arXiv  Detail & Related papers  (2025-05-29T16:15:48Z)
• PacTure: Efficient PBR Texture Generation on Packed Views with Visual Autoregressive Models [73.4445896872942]
  PacTure is a framework for generating physically-based rendering (PBR) material textures from an un-domain 3D mesh.<n>We introduce view packing, a novel technique that increases the effective resolution for each view.
  arXiv  Detail & Related papers  (2025-05-28T14:23:30Z)
• Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generation [54.588082888166504]
  We present Mogao, a unified framework that enables interleaved multi-modal generation through a causal approach.<n>Mogoo integrates a set of key technical improvements in architecture design, including a deep-fusion design, dual vision encoders, interleaved rotary position embeddings, and multi-modal classifier-free guidance.<n>Experiments show that Mogao achieves state-of-the-art performance in multi-modal understanding and text-to-image generation, but also excels in producing high-quality, coherent interleaved outputs.
  arXiv  Detail & Related papers  (2025-05-08T17:58:57Z)
• RealGeneral: Unifying Visual Generation via Temporal In-Context Learning with Video Models [22.042487298092883]
  RealGeneral is a novel framework that reformulates image generation as a conditional frame prediction task.<n>It mitigates a 14.5% improvement in subject similarity for customized generation and a 10% enhancement in image quality for canny-to-image task.
  arXiv  Detail & Related papers  (2025-03-13T14:31:52Z)
• EditAR: Unified Conditional Generation with Autoregressive Models [58.093860528672735]
  We propose EditAR, a single unified autoregressive framework for a variety of conditional image generation tasks.<n>The model takes both images and instructions as inputs, and predicts the edited images tokens in a vanilla next-token paradigm.<n>We evaluate its effectiveness across diverse tasks on established benchmarks, showing competitive performance to various state-of-the-art task-specific methods.
  arXiv  Detail & Related papers  (2025-01-08T18:59:35Z)
• Coherent and Multi-modality Image Inpainting via Latent Space Optimization [61.99406669027195]
  PILOT (intextbfPainting vtextbfIa textbfLatent textbfOptextbfTimization) is an optimization approach grounded on a novel textitsemantic centralization and textitbackground preservation loss. Our method searches latent spaces capable of generating inpainted regions that exhibit high fidelity to user-provided prompts while maintaining coherence with the background.
  arXiv  Detail & Related papers  (2024-07-10T19:58:04Z)
• MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models [34.611309081801345]
  Large diffusion-based Text-to-Image (T2I) models have shown impressive generative powers for text-to-image generation. In this paper, we propose a novel strategy to scale a generative model across new tasks with minimal compute.
  arXiv  Detail & Related papers  (2024-04-15T17:55:56Z)
• DivCon: Divide and Conquer for Progressive Text-to-Image Generation [0.0]
  Diffusion-driven text-to-image (T2I) generation has achieved remarkable advancements. layout is employed as an intermedium to bridge large language models and layout-based diffusion models. We introduce a divide-and-conquer approach which decouples the T2I generation task into simple subtasks.
  arXiv  Detail & Related papers  (2024-03-11T03:24:44Z)
• Image Inpainting via Tractable Steering of Diffusion Models [48.16994134964729]
  This paper proposes to exploit the ability of Tractable Probabilistic Models (TPMs) to exactly and efficiently compute the constrained posterior.<n>Specifically, this paper adopts a class of expressive TPMs termed Probabilistic Circuits (PCs)
  arXiv  Detail & Related papers  (2023-11-28T21:14:02Z)
• LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Model [55.20469538848806]
  LeftRefill is an innovative approach to harness large Text-to-Image (T2I) diffusion models for reference-guided image synthesis. This paper introduces LeftRefill, an innovative approach to efficiently harness large Text-to-Image (T2I) diffusion models for reference-guided image synthesis.
  arXiv  Detail & Related papers  (2023-05-19T10:29:42Z)
• Auto-regressive Image Synthesis with Integrated Quantization [55.51231796778219]
  This paper presents a versatile framework for conditional image generation. It incorporates the inductive bias of CNNs and powerful sequence modeling of auto-regression. Our method achieves superior diverse image generation performance as compared with the state-of-the-art.
  arXiv  Detail & Related papers  (2022-07-21T22:19:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.