Local Padding in Patch-Based GANs for Seamless Infinite-Sized Texture Synthesis

• URL: http://arxiv.org/abs/2309.02340v5
• Date: Thu, 07 Nov 2024 14:00:08 GMT
• Title: Local Padding in Patch-Based GANs for Seamless Infinite-Sized Texture Synthesis
• Authors: Alhasan Abdellatif, Ahmed H. Elsheikh, Hannah P. Menke,
• Abstract summary: We propose a novel approach for generating texture images at large arbitrary sizes using GANs based on patch-by-patch generation. Instead of zero-padding, the model uses textitlocal padding in the generator that shares border features between the generated patches. Our method has a significant advancement beyond existing GANs-based texture models in terms of the quality and diversity of the generated textures.
• Score: 0.8192907805418583
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Texture models based on Generative Adversarial Networks (GANs) use zero-padding to implicitly encode positional information of the image features. However, when extending the spatial input to generate images at large sizes, zero-padding can often lead to degradation in image quality due to the incorrect positional information at the center of the image. Moreover, zero-padding can limit the diversity within the generated large images. In this paper, we propose a novel approach for generating stochastic texture images at large arbitrary sizes using GANs based on patch-by-patch generation. Instead of zero-padding, the model uses \textit{local padding} in the generator that shares border features between the generated patches; providing positional context and ensuring consistency at the boundaries. The proposed models are trainable on a single texture image and have a constant GPU scalability with respect to the output image size, and hence can generate images of infinite sizes. We show in the experiments that our method has a significant advancement beyond existing GANs-based texture models in terms of the quality and diversity of the generated textures. Furthermore, the implementation of local padding in the state-of-the-art super-resolution models effectively eliminates tiling artifacts enabling large-scale super-resolution. Our code is available at \url{https://github.com/ai4netzero/Infinite_Texture_GANs}.

Related papers

• $\infty$-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions [58.42011190989414]
  We introduce a novel conditional diffusion model in infinite dimensions, $infty$-Brush for controllable large image synthesis. To our best knowledge, $infty$-Brush is the first conditional diffusion model in function space, that can controllably synthesize images at arbitrary resolutions of up to $4096times4096$ pixels.
  arXiv  Detail & Related papers  (2024-07-20T00:04:49Z)
• Infinite Texture: Text-guided High Resolution Diffusion Texture Synthesis [61.189479577198846]
  We present Infinite Texture, a method for generating arbitrarily large texture images from a text prompt. Our approach fine-tunes a diffusion model on a single texture, and learns to embed that statistical distribution in the output domain of the model. At generation time, our fine-tuned diffusion model is used through a score aggregation strategy to generate output texture images of arbitrary resolution on a single GPU.
  arXiv  Detail & Related papers  (2024-05-13T21:53:09Z)
• LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation [121.45667242282721]
  We propose a coarse-to-fine paradigm to achieve layout planning and image generation. Our proposed method outperforms the state-of-the-art models in terms of photorealistic layout and image generation.
  arXiv  Detail & Related papers  (2023-08-09T17:45:04Z)
• SDM: Spatial Diffusion Model for Large Hole Image Inpainting [106.90795513361498]
  We present a novel spatial diffusion model (SDM) that uses a few iterations to gradually deliver informative pixels to the entire image. Also, thanks to the proposed decoupled probabilistic modeling and spatial diffusion scheme, our method achieves high-quality large-hole completion.
  arXiv  Detail & Related papers  (2022-12-06T13:30:18Z)
• Delving Globally into Texture and Structure for Image Inpainting [20.954875933730808]
  Image inpainting has achieved remarkable progress and inspired abundant methods, where the critical bottleneck is identified as how to fulfill the high-frequency structure and low-frequency texture information on the masked regions with semantics. In this paper, we delve globally into texture and structure information to well capture the semantics for image inpainting. Our model is tovolution to the fashionable arts, such as Conal Neural Networks (CNNs), Attention and Transformer model, from the perspective of texture and structure information for image inpainting.
  arXiv  Detail & Related papers  (2022-09-17T02:19:26Z)
• TGHop: An Explainable, Efficient and Lightweight Method for Texture Generation [28.185787626054704]
  TGHop (an acronym of Texture Generation PixelHop) is proposed in this work. TGHop is small in its model size, mathematically transparent, efficient in training and inference, and able to generate high quality texture. It is demonstrated by experimental results that TGHop can generate texture images of superior quality with a small model size and at a fast speed.
  arXiv  Detail & Related papers  (2021-07-08T17:56:58Z)
• Controllable Person Image Synthesis with Spatially-Adaptive Warped Normalization [72.65828901909708]
  Controllable person image generation aims to produce realistic human images with desirable attributes. We introduce a novel Spatially-Adaptive Warped Normalization (SAWN), which integrates a learned flow-field to warp modulation parameters. We propose a novel self-training part replacement strategy to refine the pretrained model for the texture-transfer task.
  arXiv  Detail & Related papers  (2021-05-31T07:07:44Z)
• InfinityGAN: Towards Infinite-Resolution Image Synthesis [92.40782797030977]
  We present InfinityGAN, a method to generate arbitrary-resolution images. We show how it trains and infers patch-by-patch seamlessly with low computational resources.
  arXiv  Detail & Related papers  (2021-04-08T17:59:30Z)
• Texture Transform Attention for Realistic Image Inpainting [6.275013056564918]
  We propose a Texture Transform Attention network that better produces the missing region inpainting with fine details. Texture Transform Attention is used to create a new reassembled texture map using fine textures and coarse semantics. We evaluate our model end-to-end with the publicly available datasets CelebA-HQ and Places2.
  arXiv  Detail & Related papers  (2020-12-08T06:28:51Z)
• Texture Memory-Augmented Deep Patch-Based Image Inpainting [121.41395272974611]
  We propose a new deep inpainting framework where texture generation is guided by a texture memory of patch samples extracted from unmasked regions. The framework has a novel design that allows texture memory retrieval to be trained end-to-end with the deep inpainting network. The proposed method shows superior performance both qualitatively and quantitatively on three challenging image benchmarks.
  arXiv  Detail & Related papers  (2020-09-28T12:09:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.