Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling

• URL: http://arxiv.org/abs/2405.02941v2
• Date: Sun, 12 May 2024 18:41:09 GMT
• Title: Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling
• Authors: Jinmin Li, Tao Dai, Jingyun Zhang, Kang Liu, Jun Wang, Shaoming Wang, Shu-Tao Xia, Rizen Guo,
• Abstract summary: Recently developed generative methods, including invertible rescaling network (IRN) based and generative adversarial network (GAN) based methods, have demonstrated exceptional performance in image rescaling. However, IRN-based methods tend to produce over-smoothed results, while GAN-based methods easily generate fake details. We propose Boundary-aware Decoupled Flow Networks (BDFlow) to generate realistic and visually pleasing results.
• Score: 49.215957313126324
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently developed generative methods, including invertible rescaling network (IRN) based and generative adversarial network (GAN) based methods, have demonstrated exceptional performance in image rescaling. However, IRN-based methods tend to produce over-smoothed results, while GAN-based methods easily generate fake details, which thus hinders their real applications. To address this issue, we propose Boundary-aware Decoupled Flow Networks (BDFlow) to generate realistic and visually pleasing results. Unlike previous methods that model high-frequency information as standard Gaussian distribution directly, our BDFlow first decouples the high-frequency information into \textit{semantic high-frequency} that adheres to a Boundary distribution and \textit{non-semantic high-frequency} counterpart that adheres to a Gaussian distribution. Specifically, to capture semantic high-frequency parts accurately, we use Boundary-aware Mask (BAM) to constrain the model to produce rich textures, while non-semantic high-frequency part is randomly sampled from a Gaussian distribution.Comprehensive experiments demonstrate that our BDFlow significantly outperforms other state-of-the-art methods while maintaining lower complexity. Notably, our BDFlow improves the PSNR by 4.4 dB and the SSIM by 0.1 on average over GRAIN, utilizing only 74% of the parameters and 20% of the computation. The code will be available at https://github.com/THU-Kingmin/BAFlow.

Related papers

• Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding [84.3224556294803]
  Diffusion models excel at capturing the natural design spaces of images, molecules, DNA, RNA, and protein sequences. We aim to optimize downstream reward functions while preserving the naturalness of these design spaces. Our algorithm integrates soft value functions, which looks ahead to how intermediate noisy states lead to high rewards in the future.
  arXiv  Detail & Related papers  (2024-08-15T16:47:59Z)
• LeRF: Learning Resampling Function for Adaptive and Efficient Image Interpolation [64.34935748707673]
  Recent deep neural networks (DNNs) have made impressive progress in performance by introducing learned data priors. We propose a novel method of Learning Resampling (termed LeRF) which takes advantage of both the structural priors learned by DNNs and the locally continuous assumption. LeRF assigns spatially varying resampling functions to input image pixels and learns to predict the shapes of these resampling functions with a neural network.
  arXiv  Detail & Related papers  (2024-07-13T16:09:45Z)
• Neural Fields with Thermal Activations for Arbitrary-Scale Super-Resolution [56.089473862929886]
  We present a novel way to design neural fields such that points can be queried with an adaptive Gaussian PSF. With its theoretically guaranteed anti-aliasing, our method sets a new state of the art for arbitrary-scale single image super-resolution.
  arXiv  Detail & Related papers  (2023-11-29T14:01:28Z)
• Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion [56.38386580040991]
  Consistency Trajectory Model (CTM) is a generalization of Consistency Models (CM) CTM enables the efficient combination of adversarial training and denoising score matching loss to enhance performance. Unlike CM, CTM's access to the score function can streamline the adoption of established controllable/conditional generation methods.
  arXiv  Detail & Related papers  (2023-10-01T05:07:17Z)
• Diffusion Models with Deterministic Normalizing Flow Priors [23.212848643552395]
  We propose DiNof ($textbfDi$ffusion with $textbfNo$rmalizing $textbff$low priors), a technique that makes use of normalizing flows and diffusion models. Experiments on standard image generation datasets demonstrate the advantage of the proposed method over existing approaches.
  arXiv  Detail & Related papers  (2023-09-03T21:26:56Z)
• FFEINR: Flow Feature-Enhanced Implicit Neural Representation for Spatio-temporal Super-Resolution [4.577685231084759]
  This paper proposes a Feature-Enhanced Neural Implicit Representation (FFEINR) for super-resolution of flow field data. It can take full advantage of the implicit neural representation in terms of model structure and sampling resolution. The training process of FFEINR is facilitated by introducing feature enhancements for the input layer.
  arXiv  Detail & Related papers  (2023-08-24T02:28:18Z)
• Comparative Study of Coupling and Autoregressive Flows through Robust Statistical Tests [0.0]
  We propose an in-depth comparison of coupling and autoregressive flows, both of the affine and rational quadratic type. We focus on a set of multimodal target distributions increasing dimensionality ranging from 4 to 400. Our results indicate that the A-RQS algorithm stands out both in terms of accuracy and training speed.
  arXiv  Detail & Related papers  (2023-02-23T13:34:01Z)
• DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows [145.83812019515818]
  We propose DeFlow, a method for learning image degradations from unpaired data. We model the degradation process in the latent space of a shared flow-decoder network. We validate our DeFlow formulation on the task of joint image restoration and super-resolution.
  arXiv  Detail & Related papers  (2021-01-14T18:58:01Z)
• Deep Residual Flow for Out of Distribution Detection [27.218308616245164]
  We present a novel approach that improves upon the state-of-the-art by leveraging an expressive density model based on normalizing flows. We demonstrate the effectiveness of our method in ResNet and DenseNet architectures trained on various image datasets.
  arXiv  Detail & Related papers  (2020-01-15T16:38:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.