RETHINED: A New Benchmark and Baseline for Real-Time High-Resolution Image Inpainting On Edge Devices

• URL: http://arxiv.org/abs/2503.14757v1
• Date: Tue, 18 Mar 2025 22:02:40 GMT
• Title: RETHINED: A New Benchmark and Baseline for Real-Time High-Resolution Image Inpainting On Edge Devices
• Authors: Marcelo Sanchez, Gil Triginer, Ignacio Sarasua, Lara Raad, Coloma Ballester,
• Abstract summary: Existing image inpainting methods have shown impressive completion results for low-resolution images.<n>We propose the first baseline for REal-Time High-resolution image INpainting on Edge Devices (RETHINED) that is able to inpaint at ultra-high-resolution.
• Score: 2.1645562655776174
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing image inpainting methods have shown impressive completion results for low-resolution images. However, most of these algorithms fail at high resolutions and require powerful hardware, limiting their deployment on edge devices. Motivated by this, we propose the first baseline for REal-Time High-resolution image INpainting on Edge Devices (RETHINED) that is able to inpaint at ultra-high-resolution and can run in real-time ($\leq$ 30ms) in a wide variety of mobile devices. A simple, yet effective novel method formed by a lightweight Convolutional Neural Network (CNN) to recover structure, followed by a resolution-agnostic patch replacement mechanism to provide detailed texture. Specially our pipeline leverages the structural capacity of CNN and the high-level detail of patch-based methods, which is a key component for high-resolution image inpainting. To demonstrate the real application of our method, we conduct an extensive analysis on various mobile-friendly devices and demonstrate similar inpainting performance while being $\mathrm{100 \times faster}$ than existing state-of-the-art methods. Furthemore, we realease DF8K-Inpainting, the first free-form mask UHD inpainting dataset.

Related papers

• Bit-depth color recovery via off-the-shelf super-resolution models [4.536530093400348]
  We introduce a novel approach that integrates a super-resolution architecture to extract detailed a priori information from images.<n>We demonstrate that our approach outperforms state-of-the-art methods, highlighting the potential of super-resolution for high-fidelity color restoration.
  arXiv  Detail & Related papers  (2025-01-09T23:20:19Z)
• HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models [59.01600111737628]
  HD-Painter is a training free approach that accurately follows prompts and coherently scales to high resolution image inpainting. To this end, we design the Prompt-Aware Introverted Attention (PAIntA) layer enhancing self-attention scores. Our experiments demonstrate that HD-Painter surpasses existing state-of-the-art approaches quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2023-12-21T18:09:30Z)
• T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
  A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields. In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting. Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
  arXiv  Detail & Related papers  (2023-05-12T04:10:42Z)
• CoordFill: Efficient High-Resolution Image Inpainting via Parameterized Coordinate Querying [52.91778151771145]
  In this paper, we try to break the limitations for the first time thanks to the recent development of continuous implicit representation. Experiments show that the proposed method achieves real-time performance on the 2048$times$2048 images using a single GTX 2080 Ti GPU.
  arXiv  Detail & Related papers  (2023-03-15T11:13:51Z)
• Resolution-robust Large Mask Inpainting with Fourier Convolutions [10.152370311844445]
  Inpainting systems often struggle with large missing areas, complex geometric structures, and high-resolution images. We find that one of the main reasons for that is the lack of an effective receptive field in both the inpainting network and the loss function. We propose a new method called large mask inpainting (LaMa) to alleviate this issue.
  arXiv  Detail & Related papers  (2021-09-15T08:54:29Z)
• Spatial-Separated Curve Rendering Network for Efficient and High-Resolution Image Harmonization [59.19214040221055]
  We propose a novel spatial-separated curve rendering network (S$2$CRNet) for efficient and high-resolution image harmonization. The proposed method reduces more than 90% parameters compared with previous methods. Our method can work smoothly on higher resolution images in real-time which is more than 10$times$ faster than the existing methods.
  arXiv  Detail & Related papers  (2021-09-13T07:20:16Z)
• Generating Superpixels for High-resolution Images with Decoupled Patch Calibration [82.21559299694555]
  Patch Networks (PCNet) is designed to efficiently and accurately implement high-resolution superpixel segmentation. DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries.
  arXiv  Detail & Related papers  (2021-08-19T10:33:05Z)
• Zoom-to-Inpaint: Image Inpainting with High-Frequency Details [39.582275854002994]
  We propose applying super-resolution to coarsely reconstructed outputs, refining them at high resolution, and then downscaling the output to the original resolution. By introducing high-resolution images to the refinement network, our framework is able to reconstruct finer details that are usually smoothed out due to spectral bias. Our zoom-in, refine and zoom-out strategy, combined with high-resolution supervision and progressive learning, constitutes a framework-agnostic approach for enhancing high-frequency details.
  arXiv  Detail & Related papers  (2020-12-17T05:39:37Z)
• Efficient texture-aware multi-GAN for image inpainting [5.33024001730262]
  Recent GAN-based (Generative adversarial networks) inpainting methods show remarkable improvements. We propose a multi-GAN architecture improving both the performance and rendering efficiency.
  arXiv  Detail & Related papers  (2020-09-30T14:58:03Z)
• High-Resolution Image Inpainting with Iterative Confidence Feedback and Guided Upsampling [122.06593036862611]
  Existing image inpainting methods often produce artifacts when dealing with large holes in real applications. We propose an iterative inpainting method with a feedback mechanism. Experiments show that our method significantly outperforms existing methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2020-05-24T13:23:45Z)
• Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting [12.839962012888199]
  We propose a Contextual Residual Aggregation (CRA) mechanism that can produce high-frequency residuals for missing contents. CRA mechanism produces high-frequency residuals for missing contents by weighted aggregating residuals from contextual patches. We train the proposed model on small images with resolutions 512x512 and perform inference on high-resolution images, achieving compelling inpainting quality.
  arXiv  Detail & Related papers  (2020-05-19T18:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.