Easing Color Shifts in Score-Based Diffusion Models

• URL: http://arxiv.org/abs/2306.15832v2
• Date: Tue, 28 Nov 2023 21:18:38 GMT
• Title: Easing Color Shifts in Score-Based Diffusion Models
• Authors: Katherine Deck and Tobias Bischoff
• Abstract summary: We quantify the performance of a nonlinear bypass connection in the score network. We show that this network architecture substantially improves the resulting quality of the generated images.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Generated images of score-based models can suffer from errors in their spatial means, an effect, referred to as a color shift, which grows for larger images. This paper investigates a previously-introduced approach to mitigate color shifts in score-based diffusion models. We quantify the performance of a nonlinear bypass connection in the score network, designed to process the spatial mean of the input and to predict the mean of the score function. We show that this network architecture substantially improves the resulting quality of the generated images, and that this improvement is approximately independent of the size of the generated images. As a result, this modified architecture offers a simple solution for the color shift problem across image sizes. We additionally discuss the origin of color shifts in an idealized setting in order to motivate the approach.

Related papers

• SPDGAN: A Generative Adversarial Network based on SPD Manifold Learning for Automatic Image Colorization [1.220743263007369]
  We propose a fully automatic colorization approach based on Symmetric Positive Definite (SPD) Manifold Learning with a generative adversarial network (SPDGAN) Our model establishes an adversarial game between two discriminators and a generator. Its goal is to generate fake colorized images without losing color information across layers through residual connections.
  arXiv  Detail & Related papers  (2023-12-21T00:52:01Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Color Equivariant Convolutional Networks [50.655443383582124]
  CNNs struggle if there is data imbalance between color variations introduced by accidental recording conditions. We propose Color Equivariant Convolutions ( CEConvs), a novel deep learning building block that enables shape feature sharing across the color spectrum. We demonstrate the benefits of CEConvs in terms of downstream performance to various tasks and improved robustness to color changes, including train-test distribution shifts.
  arXiv  Detail & Related papers  (2023-10-30T09:18:49Z)
• Distance Weighted Trans Network for Image Completion [52.318730994423106]
  We propose a new architecture that relies on Distance-based Weighted Transformer (DWT) to better understand the relationships between an image's components. CNNs are used to augment the local texture information of coarse priors. DWT blocks are used to recover certain coarse textures and coherent visual structures.
  arXiv  Detail & Related papers  (2023-10-11T12:46:11Z)
• DARC: Distribution-Aware Re-Coloring Model for Generalizable Nucleus Segmentation [68.43628183890007]
  We argue that domain gaps can also be caused by different foreground (nucleus)-background ratios. First, we introduce a re-coloring method that relieves dramatic image color variations between different domains. Second, we propose a new instance normalization method that is robust to the variation in the foreground-background ratios.
  arXiv  Detail & Related papers  (2023-09-01T01:01:13Z)
• Estimating Appearance Models for Image Segmentation via Tensor Factorization [0.0]
  We propose a new approach to directly estimate appearance models from the image without prior information on the underlying segmentation. Our method uses local high order color statistics from the image as an input to tensor factorization-based estimator for latent variable models. This approach is able to estimate models in multiregion images and automatically output the regions proportions without prior user interaction.
  arXiv  Detail & Related papers  (2022-08-16T17:21:00Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• High-dimensional Assisted Generative Model for Color Image Restoration [12.459091135428885]
  This work presents an unsupervised deep learning scheme that exploits high-dimensional assisted score-based generative model for color image restoration tasks. Considering the sample number and internal dimension in score-based generative model, two different high-dimensional ways are proposed: The channel-copy transformation increases the sample number and the pixel-scale transformation decreases feasible dimension space. To alleviate the difficulty of learning high-dimensional representation, a progressive strategy is proposed to leverage the performance.
  arXiv  Detail & Related papers  (2021-08-14T04:05:29Z)
• Wavelet Transform-assisted Adaptive Generative Modeling for Colorization [15.814591440291652]
  This study presents a novel scheme that exploiting the score-based generative model in wavelet domain to address the issue. By taking advantage of the multi-scale and multi-channel representation via wavelet transform, the proposed model learns the priors from stacked wavelet coefficient components. Experiments demonstrated remarkable improvements of the proposed model on colorization quality, particularly on colorization robustness and diversity.
  arXiv  Detail & Related papers  (2021-07-09T07:12:39Z)
• Image Inpainting with Learnable Feature Imputation [8.293345261434943]
  A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. We propose (layer-wise) feature imputation of the missing input values to a convolution. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model.
  arXiv  Detail & Related papers  (2020-11-02T16:05:32Z)
• Image Fine-grained Inpainting [89.17316318927621]
  We present a one-stage model that utilizes dense combinations of dilated convolutions to obtain larger and more effective receptive fields. To better train this efficient generator, except for frequently-used VGG feature matching loss, we design a novel self-guided regression loss. We also employ a discriminator with local and global branches to ensure local-global contents consistency.
  arXiv  Detail & Related papers  (2020-02-07T03:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.