CoordFill: Efficient High-Resolution Image Inpainting via Parameterized
Coordinate Querying
- URL: http://arxiv.org/abs/2303.08524v1
- Date: Wed, 15 Mar 2023 11:13:51 GMT
- Title: CoordFill: Efficient High-Resolution Image Inpainting via Parameterized
Coordinate Querying
- Authors: Weihuang Liu, Xiaodong Cun, Chi-Man Pun, Menghan Xia, Yong Zhang, Jue
Wang
- Abstract summary: In this paper, we try to break the limitations for the first time thanks to the recent development of continuous implicit representation.
Experiments show that the proposed method achieves real-time performance on the 2048$times$2048 images using a single GTX 2080 Ti GPU.
- Score: 52.91778151771145
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image inpainting aims to fill the missing hole of the input. It is hard to
solve this task efficiently when facing high-resolution images due to two
reasons: (1) Large reception field needs to be handled for high-resolution
image inpainting. (2) The general encoder and decoder network synthesizes many
background pixels synchronously due to the form of the image matrix. In this
paper, we try to break the above limitations for the first time thanks to the
recent development of continuous implicit representation. In detail, we
down-sample and encode the degraded image to produce the spatial-adaptive
parameters for each spatial patch via an attentional Fast Fourier
Convolution(FFC)-based parameter generation network. Then, we take these
parameters as the weights and biases of a series of multi-layer
perceptron(MLP), where the input is the encoded continuous coordinates and the
output is the synthesized color value. Thanks to the proposed structure, we
only encode the high-resolution image in a relatively low resolution for larger
reception field capturing. Then, the continuous position encoding will be
helpful to synthesize the photo-realistic high-frequency textures by
re-sampling the coordinate in a higher resolution. Also, our framework enables
us to query the coordinates of missing pixels only in parallel, yielding a more
efficient solution than the previous methods. Experiments show that the
proposed method achieves real-time performance on the 2048$\times$2048 images
using a single GTX 2080 Ti GPU and can handle 4096$\times$4096 images, with
much better performance than existing state-of-the-art methods visually and
numerically. The code is available at:
https://github.com/NiFangBaAGe/CoordFill.
Related papers
- Transformer based Pluralistic Image Completion with Reduced Information Loss [72.92754600354199]
Transformer based methods have achieved great success in image inpainting recently.
They regard each pixel as a token, thus suffering from an information loss issue.
We propose a new transformer based framework called "PUT"
arXiv Detail & Related papers (2024-03-31T01:20:16Z) - T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields.
In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting.
Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
arXiv Detail & Related papers (2023-05-12T04:10:42Z) - CUF: Continuous Upsampling Filters [25.584630142930123]
In this paper, we consider one of the most important operations in image processing: upsampling.
We propose to parameterize upsampling kernels as neural fields.
This parameterization leads to a compact architecture that obtains a 40-fold reduction in the number of parameters when compared with competing arbitrary-scale super-resolution architectures.
arXiv Detail & Related papers (2022-10-13T12:45:51Z) - Spatial-Separated Curve Rendering Network for Efficient and
High-Resolution Image Harmonization [59.19214040221055]
We propose a novel spatial-separated curve rendering network (S$2$CRNet) for efficient and high-resolution image harmonization.
The proposed method reduces more than 90% parameters compared with previous methods.
Our method can work smoothly on higher resolution images in real-time which is more than 10$times$ faster than the existing methods.
arXiv Detail & Related papers (2021-09-13T07:20:16Z) - InfinityGAN: Towards Infinite-Resolution Image Synthesis [92.40782797030977]
We present InfinityGAN, a method to generate arbitrary-resolution images.
We show how it trains and infers patch-by-patch seamlessly with low computational resources.
arXiv Detail & Related papers (2021-04-08T17:59:30Z) - Spatially-Adaptive Pixelwise Networks for Fast Image Translation [57.359250882770525]
We introduce a new generator architecture, aimed at fast and efficient high-resolution image-to-image translation.
We use pixel-wise networks; that is, each pixel is processed independently of others.
Our model is up to 18x faster than state-of-the-art baselines.
arXiv Detail & Related papers (2020-12-05T10:02:03Z) - Contextual Residual Aggregation for Ultra High-Resolution Image
Inpainting [12.839962012888199]
We propose a Contextual Residual Aggregation (CRA) mechanism that can produce high-frequency residuals for missing contents.
CRA mechanism produces high-frequency residuals for missing contents by weighted aggregating residuals from contextual patches.
We train the proposed model on small images with resolutions 512x512 and perform inference on high-resolution images, achieving compelling inpainting quality.
arXiv Detail & Related papers (2020-05-19T18:55:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.