Neural Fields with Thermal Activations for Arbitrary-Scale Super-Resolution

• URL: http://arxiv.org/abs/2311.17643v2
• Date: Thu, 14 Mar 2024 19:17:34 GMT
• Title: Neural Fields with Thermal Activations for Arbitrary-Scale Super-Resolution
• Authors: Alexander Becker, Rodrigo Caye Daudt, Nando Metzger, Jan Dirk Wegner, Konrad Schindler,
• Abstract summary: We present a novel way to design neural fields such that points can be queried with an adaptive Gaussian PSF. With its theoretically guaranteed anti-aliasing, our method sets a new state of the art for arbitrary-scale single image super-resolution.
• Score: 56.089473862929886
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent approaches for arbitrary-scale single image super-resolution (ASSR) have used local neural fields to represent continuous signals that can be sampled at arbitrary rates. However, the point-wise query of the neural field does not naturally match the point spread function (PSF) of a given pixel, which may cause aliasing in the super-resolved image. We present a novel way to design neural fields such that points can be queried with an adaptive Gaussian PSF, so as to guarantee correct anti-aliasing at any desired output resolution. We achieve this with a novel activation function derived from Fourier theory. Querying points with a Gaussian PSF, compliant with sampling theory, does not incur any additional computational cost in our framework, unlike filtering in the image domain. With its theoretically guaranteed anti-aliasing, our method sets a new state of the art for ASSR, while being more parameter-efficient than previous methods. Notably, even a minimal version of our model still outperforms previous methods in most cases, while adding 2-4 orders of magnitude fewer parameters. Code and pretrained models are available at https://github.com/prs-eth/thera.

Related papers

• Adaptive Random Fourier Features Training Stabilized By Resampling With Applications in Image Regression [0.8947831206263182]
  We present an enhanced adaptive random Fourier features (ARFF) training algorithm for shallow neural networks. This method uses a particle filter type resampling technique to stabilize the training process and reduce sensitivity to parameter choices.
  arXiv  Detail & Related papers  (2024-10-08T22:08:03Z)
• GaussianSR: High Fidelity 2D Gaussian Splatting for Arbitrary-Scale Image Super-Resolution [29.49617080140511]
  Implicit neural representations (INRs) have significantly advanced the field of arbitrary-scale super-resolution (ASSR) of images. Most existing INR-based ASSR networks first extract features from the given low-resolution image using an encoder, and then render the super-resolved result via a multi-layer perceptron decoder. We propose a novel ASSR method named GaussianSR that overcomes this limitation through 2D Gaussian Splatting (2DGS)
  arXiv  Detail & Related papers  (2024-07-25T13:53:48Z)
• Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling [49.215957313126324]
  Recently developed generative methods, including invertible rescaling network (IRN) based and generative adversarial network (GAN) based methods, have demonstrated exceptional performance in image rescaling. However, IRN-based methods tend to produce over-smoothed results, while GAN-based methods easily generate fake details. We propose Boundary-aware Decoupled Flow Networks (BDFlow) to generate realistic and visually pleasing results.
  arXiv  Detail & Related papers  (2024-05-05T14:05:33Z)
• CorresNeRF: Image Correspondence Priors for Neural Radiance Fields [45.40164120559542]
  CorresNeRF is a novel method that leverages image correspondence priors computed by off-the-shelf methods to supervise NeRF training. We show that this simple yet effective technique of using correspondence priors can be applied as a plug-and-play module across different NeRF variants.
  arXiv  Detail & Related papers  (2023-12-11T18:55:29Z)
• FOVEA: Foveated Image Magnification for Autonomous Navigation [53.69803081925454]
  We propose an attentional approach that elastically magnifies certain regions while maintaining a small input canvas. Our proposed method boosts the detection AP over standard Faster R-CNN, with and without finetuning. On the autonomous driving datasets Argoverse-HD and BDD100K, we show our proposed method boosts the detection AP over standard Faster R-CNN, with and without finetuning.
  arXiv  Detail & Related papers  (2021-08-27T03:07:55Z)
• Semi-Sparsity for Smoothing Filters [1.1404527665142667]
  We show a new semi-sparsity smoothing algorithm based on a novel sparsity-inducing framework. We show many benefits to a series of signal/image processing and computer vision applications.
  arXiv  Detail & Related papers  (2021-07-01T17:31:42Z)
• Sigma-Delta and Distributed Noise-Shaping Quantization Methods for Random Fourier Features [73.25551965751603]
  We prove that our quantized RFFs allow a high accuracy approximation of the underlying kernels. We show that the quantized RFFs can be further compressed, yielding an excellent trade-off between memory use and accuracy. We empirically show by testing the performance of our methods on several machine learning tasks that our method compares favorably to other state of the art quantization methods in this context.
  arXiv  Detail & Related papers  (2021-06-04T17:24:47Z)
• UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution [74.82282301089994]
  In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions. We show that spatial encoding is indeed a missing key towards the next-stage high-accuracy implicit image function. Our UltraSR sets new state-of-the-art performance on the DIV2K benchmark under all super-resolution scales.
  arXiv  Detail & Related papers  (2021-03-23T17:36:42Z)
• Image Inpainting with Learnable Feature Imputation [8.293345261434943]
  A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. We propose (layer-wise) feature imputation of the missing input values to a convolution. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model.
  arXiv  Detail & Related papers  (2020-11-02T16:05:32Z)
• End-to-end Interpretable Learning of Non-blind Image Deblurring [102.75982704671029]
  Non-blind image deblurring is typically formulated as a linear least-squares problem regularized by natural priors on the corresponding sharp picture's gradients. We propose to precondition the Richardson solver using approximate inverse filters of the (known) blur and natural image prior kernels.
  arXiv  Detail & Related papers  (2020-07-03T15:45:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.