ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation

• URL: http://arxiv.org/abs/2311.18822v3
• Date: Mon, 10 Mar 2025 17:56:21 GMT
• Title: ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation
• Abstract summary: ElasticDiffusion is a training-free decoding method that enables pretrained text-to-image diffusion models to generate images with various sizes.<n>Our experiments and qualitative results show superior image coherence quality across aspect ratios compared to MultiDiffusion and the standard decoding strategy of Stable Diffusion.
• Score: 15.057394130958583
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Diffusion models have revolutionized image generation in recent years, yet they are still limited to a few sizes and aspect ratios. We propose ElasticDiffusion, a novel training-free decoding method that enables pretrained text-to-image diffusion models to generate images with various sizes. ElasticDiffusion attempts to decouple the generation trajectory of a pretrained model into local and global signals. The local signal controls low-level pixel information and can be estimated on local patches, while the global signal is used to maintain overall structural consistency and is estimated with a reference image. We test our method on CelebA-HQ (faces) and LAION-COCO (objects/indoor/outdoor scenes). Our experiments and qualitative results show superior image coherence quality across aspect ratios compared to MultiDiffusion and the standard decoding strategy of Stable Diffusion. Project page: https://elasticdiffusion.github.io/

Related papers

• A Frozen Pixel-Space Diffusion Model Can Guide Itself with Its Own Samples [53.8898272536272]
  Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels.<n>Recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures.<n>We show there is a cheaper, complementary strategy: textbfa frozen, pretrained pixel diffusion model can guide itself
  arXiv  Detail & Related papers  (2026-07-31T07:52:08Z)
• Scale Space Diffusion [46.993305407761675]
  Scale-space theory exhibits a similar hierarchy via low-pass filtering.<n>We formalize this connection and show that highly noisy diffusion states contain no more information than small, downsampled images.<n>We introduce Flexi-UNet, a UNet variant that performs resolution-preserving and resolution-increasing denoising using only the necessary parts of the network.
  arXiv  Detail & Related papers  (2026-03-09T17:59:42Z)
• Ambient Diffusion Omni: Training Good Models with Bad Data [45.821861121026394]
  We show how to use low-quality, synthetic, and out-of-distribution images to improve the quality of a diffusion model.<n>We present Ambient Omni, a principled framework to train diffusion models that can extract signal from all available images.
  arXiv  Detail & Related papers  (2025-06-10T22:37:39Z)
• Context-guided Responsible Data Augmentation with Diffusion Models [29.41191005466334]
  We propose a text-to-image (T2I) data augmentation method, named DiffCoRe-Mix, that computes a set of generative counterparts for a training sample. To preserve key semantic axes, we also filter out undesired generative samples in our augmentation process. We extensively evaluate our technique on ImageNet-1K,Tiny ImageNet-200, CIFAR-100, Flowers102, CUB-Birds, Stanford Cars, and Caltech datasets.
  arXiv  Detail & Related papers  (2025-03-12T00:12:27Z)
• Gen-SIS: Generative Self-augmentation Improves Self-supervised Learning [52.170253590364545]
  Gen-SIS is a diffusion-based augmentation technique trained exclusively on unlabeled image data. We show that these self-augmentations', i.e. generative augmentations based on the vanilla SSL encoder embeddings, facilitate the training of a stronger SSL encoder.
  arXiv  Detail & Related papers  (2024-12-02T16:20:59Z)
• Fast constrained sampling in pre-trained diffusion models [80.99262780028015]
  We propose an algorithm that enables fast, high-quality generation under arbitrary constraints.<n>Our approach produces results that rival or surpass the state-of-the-art training-free inference methods.
  arXiv  Detail & Related papers  (2024-10-24T14:52:38Z)
• Sparse Repellency for Shielded Generation in Text-to-image Diffusion Models [29.083402085790016]
  We propose a method that coaxes the sampled trajectories of pretrained diffusion models to land on images that fall outside of a reference set. We achieve this by adding repellency terms to the diffusion SDE throughout the generation trajectory. We show that adding SPELL to popular diffusion models improves their diversity while impacting their FID only marginally, and performs comparatively better than other recent training-free diversity methods.
  arXiv  Detail & Related papers  (2024-10-08T13:26:32Z)
• Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment [56.609042046176555]
  suboptimal noise-data mapping leads to slow training of diffusion models. Drawing inspiration from the immiscibility phenomenon in physics, we propose Immiscible Diffusion. Our approach is remarkably simple, requiring only one line of code to restrict the diffuse-able area for each image.
  arXiv  Detail & Related papers  (2024-06-18T06:20:42Z)
• FreeSeg-Diff: Training-Free Open-Vocabulary Segmentation with Diffusion Models [56.71672127740099]
  We focus on the task of image segmentation, which is traditionally solved by training models on closed-vocabulary datasets. We leverage different and relatively small-sized, open-source foundation models for zero-shot open-vocabulary segmentation. Our approach (dubbed FreeSeg-Diff), which does not rely on any training, outperforms many training-based approaches on both Pascal VOC and COCO datasets.
  arXiv  Detail & Related papers  (2024-03-29T10:38:25Z)
• Improving Diffusion-Based Image Synthesis with Context Prediction [49.186366441954846]
  Existing diffusion models mainly try to reconstruct input image from a corrupted one with a pixel-wise or feature-wise constraint along spatial axes. We propose ConPreDiff to improve diffusion-based image synthesis with context prediction. Our ConPreDiff consistently outperforms previous methods and achieves a new SOTA text-to-image generation results on MS-COCO, with a zero-shot FID score of 6.21.
  arXiv  Detail & Related papers  (2024-01-04T01:10:56Z)
• PanGu-Draw: Advancing Resource-Efficient Text-to-Image Synthesis with Time-Decoupled Training and Reusable Coop-Diffusion [45.06392070934473]
  "PanGu-Draw" is a novel latent diffusion model designed for resource-efficient text-to-image synthesis. We introduce "Coop-Diffusion", an algorithm that enables the cooperative use of various pre-trained diffusion models. Empirical validations of Pangu-Draw show its exceptional prowess in text-to-image and multi-control image generation.
  arXiv  Detail & Related papers  (2023-12-27T09:21:45Z)
• Learned representation-guided diffusion models for large-image generation [58.192263311786824]
  We introduce a novel approach that trains diffusion models conditioned on embeddings from self-supervised learning (SSL) Our diffusion models successfully project these features back to high-quality histopathology and remote sensing images. Augmenting real data by generating variations of real images improves downstream accuracy for patch-level and larger, image-scale classification tasks.
  arXiv  Detail & Related papers  (2023-12-12T14:45:45Z)
• Diffusion in Diffusion: Cyclic One-Way Diffusion for Text-Vision-Conditioned Generation [11.80682025950519]
  In this work, we investigate the diffusion (physics) in diffusion (machine learning) properties. We propose our Cyclic One-Way Diffusion (COW) method to control the direction of diffusion phenomenon. Our method provides a novel perspective to understand the task needs and is applicable to a wider range of customization scenarios.
  arXiv  Detail & Related papers  (2023-06-14T05:25:06Z)
• Your Diffusion Model is Secretly a Zero-Shot Classifier [90.40799216880342]
  We show that density estimates from large-scale text-to-image diffusion models can be leveraged to perform zero-shot classification. Our generative approach to classification attains strong results on a variety of benchmarks. Our results are a step toward using generative over discriminative models for downstream tasks.
  arXiv  Detail & Related papers  (2023-03-28T17:59:56Z)
• GECCO: Geometrically-Conditioned Point Diffusion Models [60.28388617034254]
  Diffusion models generating images conditionally on text have recently made a splash far beyond the computer vision community. Here, we tackle the related problem of generating point clouds, both unconditionally, and conditionally with images. For the latter, we introduce a novel geometrically-motivated conditioning scheme based on projecting sparse image features into the point cloud.
  arXiv  Detail & Related papers  (2023-03-10T13:45:44Z)
• Lossy Image Compression with Conditional Diffusion Models [25.158390422252097]
  This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models. In contrast to VAE-based neural compression, where the (mean) decoder is a deterministic neural network, our decoder is a conditional diffusion model. Our approach yields stronger reported FID scores than the GAN-based model, while also yielding competitive performance with VAE-based models in several distortion metrics.
  arXiv  Detail & Related papers  (2022-09-14T21:53:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.