DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance

• URL: http://arxiv.org/abs/2406.18459v5
• Date: Tue, 27 Aug 2024 05:46:06 GMT
• Title: DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance
• Authors: Younghyun Kim, Geunmin Hwang, Junyu Zhang, Eunbyung Park,
• Abstract summary: Large-scale generative models, such as text-to-image diffusion models, have garnered widespread attention across diverse domains. Existing large-scale diffusion models are confined to generating images of up to 1K resolution. We propose a novel progressive approach that fully utilizes generated low-resolution images to guide the generation of higher-resolution images.
• Score: 11.44012694656102
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-scale generative models, such as text-to-image diffusion models, have garnered widespread attention across diverse domains due to their creative and high-fidelity image generation. Nonetheless, existing large-scale diffusion models are confined to generating images of up to 1K resolution, which is far from meeting the demands of contemporary commercial applications. Directly sampling higher-resolution images often yields results marred by artifacts such as object repetition and distorted shapes. Addressing the aforementioned issues typically necessitates training or fine-tuning models on higher-resolution datasets. However, this poses a formidable challenge due to the difficulty in collecting large-scale high-resolution images and substantial computational resources. While several preceding works have proposed alternatives to bypass the cumbersome training process, they often fail to produce convincing results. In this work, we probe the generative ability of diffusion models at higher resolution beyond their original capability and propose a novel progressive approach that fully utilizes generated low-resolution images to guide the generation of higher-resolution images. Our method obviates the need for additional training or fine-tuning which significantly lowers the burden of computational costs. Extensive experiments and results validate the efficiency and efficacy of our method. Project page: https://yhyun225.github.io/DiffuseHigh/

Related papers

• UltraPixel: Advancing Ultra-High-Resolution Image Synthesis to New Peaks [36.61645124563195]
  We present UltraPixel, a novel architecture utilizing cascade diffusion models to generate high-quality images at multiple resolutions. We use semantics-rich representations of lower-resolution images in the later denoising stage to guide the whole generation of highly detailed high-resolution images. Our model achieves fast training with reduced data requirements, producing photo-realistic high-resolution images.
  arXiv  Detail & Related papers  (2024-07-02T11:02:19Z)
• CasSR: Activating Image Power for Real-World Image Super-Resolution [24.152495730507823]
  Cascaded diffusion for Super-Resolution, CasSR, is a novel method designed to produce highly detailed and realistic images. We develop a cascaded controllable diffusion model that aims to optimize the extraction of information from low-resolution images.
  arXiv  Detail & Related papers  (2024-03-18T03:59:43Z)
• Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation [112.08287900261898]
  This paper proposes a novel self-cascade diffusion model for rapid adaptation to higher-resolution image and video generation. Our approach achieves a 5X training speed-up and requires only an additional 0.002M tuning parameters. Experiments demonstrate that our approach can quickly adapt to higher resolution image and video synthesis by fine-tuning for just 10k steps, with virtually no additional inference time.
  arXiv  Detail & Related papers  (2024-02-16T07:48:35Z)
• ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models [126.35334860896373]
  We investigate the capability of generating images from pre-trained diffusion models at much higher resolutions than the training image sizes. Existing works for higher-resolution generation, such as attention-based and joint-diffusion approaches, cannot well address these issues. We propose a simple yet effective re-dilation that can dynamically adjust the convolutional perception field during inference.
  arXiv  Detail & Related papers  (2023-10-11T17:52:39Z)
• CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation [49.3016007471979]
  Large generative diffusion models have revolutionized text-to-image generation and offer immense potential for conditional generation tasks. However, their widespread adoption is hindered by the high computational cost, which limits their real-time application. We introduce a novel method dubbed CoDi, that adapts a pre-trained latent diffusion model to accept additional image conditioning inputs.
  arXiv  Detail & Related papers  (2023-10-02T17:59:18Z)
• Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis [62.07413805483241]
  Steered Diffusion is a framework for zero-shot conditional image generation using a diffusion model trained for unconditional generation. We present experiments using steered diffusion on several tasks including inpainting, colorization, text-guided semantic editing, and image super-resolution.
  arXiv  Detail & Related papers  (2023-09-30T02:03:22Z)
• ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolution [84.73658185158222]
  We propose a diffusion model-based super-resolution method called ACDMSR. Our method adapts the standard diffusion model to perform super-resolution through a deterministic iterative denoising process. Our approach generates more visually realistic counterparts for low-resolution images, emphasizing its effectiveness in practical scenarios.
  arXiv  Detail & Related papers  (2023-07-03T06:49:04Z)
• A Generic Approach for Enhancing GANs by Regularized Latent Optimization [79.00740660219256]
  We introduce a generic framework called em generative-model inference that is capable of enhancing pre-trained GANs effectively and seamlessly. Our basic idea is to efficiently infer the optimal latent distribution for the given requirements using Wasserstein gradient flow techniques.
  arXiv  Detail & Related papers  (2021-12-07T05:22:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.