Dynamic Implicit Image Function for Efficient Arbitrary-Scale Image Representation

• URL: http://arxiv.org/abs/2306.12321v2
• Date: Mon, 25 Sep 2023 14:05:50 GMT
• Title: Dynamic Implicit Image Function for Efficient Arbitrary-Scale Image Representation
• Authors: Zongyao He, Zhi Jin
• Abstract summary: We propose Dynamic Implicit Image Function (DIIF), which is a fast and efficient method to represent images with arbitrary resolution. We propose a coordinate grouping and slicing strategy, which enables the neural network to perform decoding from coordinate slices to pixel value slices. With dynamic coordinate slicing, DIIF significantly reduces the computational cost when encountering arbitrary-scale SR.
• Score: 24.429100808481394
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent years have witnessed the remarkable success of implicit neural representation methods. The recent work Local Implicit Image Function (LIIF) has achieved satisfactory performance for continuous image representation, where pixel values are inferred from a neural network in a continuous spatial domain. However, the computational cost of such implicit arbitrary-scale super-resolution (SR) methods increases rapidly as the scale factor increases, which makes arbitrary-scale SR time-consuming. In this paper, we propose Dynamic Implicit Image Function (DIIF), which is a fast and efficient method to represent images with arbitrary resolution. Instead of taking an image coordinate and the nearest 2D deep features as inputs to predict its pixel value, we propose a coordinate grouping and slicing strategy, which enables the neural network to perform decoding from coordinate slices to pixel value slices. We further propose a Coarse-to-Fine Multilayer Perceptron (C2F-MLP) to perform decoding with dynamic coordinate slicing, where the number of coordinates in each slice varies as the scale factor varies. With dynamic coordinate slicing, DIIF significantly reduces the computational cost when encountering arbitrary-scale SR. Experimental results demonstrate that DIIF can be integrated with implicit arbitrary-scale SR methods and achieves SOTA SR performance with significantly superior computational efficiency, thereby opening a path for real-time arbitrary-scale image representation. Our code can be found at https://github.com/HeZongyao/DIIF.

Related papers

• T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
  A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields. In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting. Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
  arXiv  Detail & Related papers  (2023-05-12T04:10:42Z)
• CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution [158.2282163651066]
  This paper proposes a continuous implicit attention-in-attention network, called CiaoSR. We explicitly design an implicit attention network to learn the ensemble weights for the nearby local features. We embed a scale-aware attention in this implicit attention network to exploit additional non-local information.
  arXiv  Detail & Related papers  (2022-12-08T15:57:46Z)
• Effective Invertible Arbitrary Image Rescaling [77.46732646918936]
  Invertible Neural Networks (INN) are able to increase upscaling accuracy significantly by optimizing the downscaling and upscaling cycle jointly. A simple and effective invertible arbitrary rescaling network (IARN) is proposed to achieve arbitrary image rescaling by training only one model in this work. It is shown to achieve a state-of-the-art (SOTA) performance in bidirectional arbitrary rescaling without compromising perceptual quality in LR outputs.
  arXiv  Detail & Related papers  (2022-09-26T22:22:30Z)
• Adaptive Local Implicit Image Function for Arbitrary-scale Super-resolution [61.95533972380704]
  Local implicit image function (LIIF) denotes images as a continuous function where pixel values are expansion by using the corresponding coordinates as inputs. LIIF can be adopted for arbitrary-scale image super-resolution tasks, resulting in a single effective and efficient model for various up-scaling factors. We propose a novel adaptive local image function (A-LIIF) to alleviate this problem.
  arXiv  Detail & Related papers  (2022-08-07T11:23:23Z)
• UHD Image Deblurring via Multi-scale Cubic-Mixer [12.402054374952485]
  transformer-based algorithms are making a splash in the domain of image deblurring. These algorithms depend on the self-attention mechanism with CNN stem to model long range dependencies between tokens.
  arXiv  Detail & Related papers  (2022-06-08T05:04:43Z)
• Hybrid Pixel-Unshuffled Network for Lightweight Image Super-Resolution [64.54162195322246]
  Convolutional neural network (CNN) has achieved great success on image super-resolution (SR) Most deep CNN-based SR models take massive computations to obtain high performance. We propose a novel Hybrid Pixel-Unshuffled Network (HPUN) by introducing an efficient and effective downsampling module into the SR task.
  arXiv  Detail & Related papers  (2022-03-16T20:10:41Z)
• Enhancing Multi-Scale Implicit Learning in Image Super-Resolution with Integrated Positional Encoding [4.781615891172263]
  We consider each pixel as the aggregation of signals from a local area in an image super-resolution context. We propose integrated positional encoding (IPE), extending traditional positional encoding by aggregating frequency information over the pixel area. We show the effectiveness of IPE-LIIF by quantitative and qualitative evaluations, and further demonstrate the generalization ability of IPE to larger image scales.
  arXiv  Detail & Related papers  (2021-12-10T06:09:55Z)
• Local Texture Estimator for Implicit Representation Function [10.165529175855712]
  Local Texture Estimator (LTE) is a dominant-frequency estimator for natural images. LTE is capable of characterizing image textures in 2D Fourier space. We show that an LTE-based neural function outperforms existing deep SR methods within an arbitrary-scale.
  arXiv  Detail & Related papers  (2021-11-17T06:01:17Z)
• UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution [74.82282301089994]
  In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions. We show that spatial encoding is indeed a missing key towards the next-stage high-accuracy implicit image function. Our UltraSR sets new state-of-the-art performance on the DIV2K benchmark under all super-resolution scales.
  arXiv  Detail & Related papers  (2021-03-23T17:36:42Z)
• Tone Mapping Based on Multi-scale Histogram Synthesis [6.6399785438250705]
  We present a novel tone mapping algorithm that can be used for displaying wide dynamic range (WDR) images on low dynamic range (LDR) devices. The proposed algorithm is mainly motivated by the logarithmic response and local adaptation features of the human visual system. Experimental results show that the proposed algorithm can generate high brightness, good contrast, and appealing images.
  arXiv  Detail & Related papers  (2021-01-31T08:11:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.