Real-Time Image Demoireing on Mobile Devices

• URL: http://arxiv.org/abs/2302.02184v1
• Date: Sat, 4 Feb 2023 15:42:42 GMT
• Title: Real-Time Image Demoireing on Mobile Devices
• Authors: Yuxin Zhang, Mingbao Lin, Xunchao Li, Han Liu, Guozhi Wang, Fei Chao, Shuai Ren, Yafei Wen, Xiaoxin Chen, Rongrong Ji
• Abstract summary: We propose a dynamic demoireing acceleration method (DDA) towards a real-time deployment on mobile devices. Our stimulus stems from a simple-yet-universal fact that moire patterns often unbalancedly distribute across an image. Our method can drastically reduce the inference time, leading to a real-time image demoireing on mobile devices.
• Score: 59.59997851375429
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Moire patterns appear frequently when taking photos of digital screens, drastically degrading the image quality. Despite the advance of CNNs in image demoireing, existing networks are with heavy design, causing redundant computation burden for mobile devices. In this paper, we launch the first study on accelerating demoireing networks and propose a dynamic demoireing acceleration method (DDA) towards a real-time deployment on mobile devices. Our stimulus stems from a simple-yet-universal fact that moire patterns often unbalancedly distribute across an image. Consequently, excessive computation is wasted upon non-moire areas. Therefore, we reallocate computation costs in proportion to the complexity of image patches. In order to achieve this aim, we measure the complexity of an image patch by designing a novel moire prior that considers both colorfulness and frequency information of moire patterns. Then, we restore image patches with higher-complexity using larger networks and the ones with lower-complexity are assigned with smaller networks to relieve the computation burden. At last, we train all networks in a parameter-shared supernet paradigm to avoid additional parameter burden. Extensive experiments on several benchmarks demonstrate the efficacy of our proposed DDA. In addition, the acceleration evaluated on the VIVO X80 Pro smartphone equipped with a chip of Snapdragon 8 Gen 1 shows that our method can drastically reduce the inference time, leading to a real-time image demoireing on mobile devices. Source codes and models are released at https://github.com/zyxxmu/DDA

Related papers

• Zero-Shot Detection of AI-Generated Images [54.01282123570917]
  We propose a zero-shot entropy-based detector (ZED) to detect AI-generated images. Inspired by recent works on machine-generated text detection, our idea is to measure how surprising the image under analysis is compared to a model of real images. ZED achieves an average improvement of more than 3% over the SoTA in terms of accuracy.
  arXiv  Detail & Related papers  (2024-09-24T08:46:13Z)
• Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models [26.926712014346432]
  This paper presents innovative enhancements to diffusion models by integrating a novel multi-resolution network and time-dependent layer normalization. Our method's efficacy is demonstrated on the class-conditional ImageNet generation benchmark, setting new state-of-the-art FID scores of 1.70 on ImageNet 256 x 256 and 2.89 on ImageNet 512 x 512.
  arXiv  Detail & Related papers  (2024-06-13T17:59:58Z)
• MOFA: A Model Simplification Roadmap for Image Restoration on Mobile Devices [17.54747506334433]
  We propose a roadmap that can be applied to further accelerate image restoration models prior to deployment. Our approach decreases runtime by up to 13% and reduces the number of parameters by up to 23%, while increasing PSNR and SSIM.
  arXiv  Detail & Related papers  (2023-08-24T01:29:15Z)
• T-former: An Efficient Transformer for Image Inpainting [50.43302925662507]
  A class of attention-based network architectures, called transformer, has shown significant performance on natural language processing fields. In this paper, we design a novel attention linearly related to the resolution according to Taylor expansion, and based on this attention, a network called $T$-former is designed for image inpainting. Experiments on several benchmark datasets demonstrate that our proposed method achieves state-of-the-art accuracy while maintaining a relatively low number of parameters and computational complexity.
  arXiv  Detail & Related papers  (2023-05-12T04:10:42Z)
• Real-Time Neural Light Field on Mobile Devices [54.44982318758239]
  We introduce a novel network architecture that runs efficiently on mobile devices with low latency and small size. Our model achieves high-resolution generation while maintaining real-time inference for both synthetic and real-world scenes.
  arXiv  Detail & Related papers  (2022-12-15T18:58:56Z)
• FMD-cGAN: Fast Motion Deblurring using Conditional Generative Adversarial Networks [26.878173373199786]
  We present a Fast Motion Deblurring-Conditional Generative Adversarial Network (FMD-cGAN) that helps in blind motion deblurring of a single image. FMD-cGAN delivers impressive structural similarity and visual appearance after deblurring an image.
  arXiv  Detail & Related papers  (2021-11-30T14:30:44Z)
• Overparametrization of HyperNetworks at Fixed FLOP-Count Enables Fast Neural Image Enhancement [0.0]
  Deep convolutional neural networks can enhance images taken with small mobile camera sensors and excel at tasks like demoisaicing, denoising and super-resolution. For practical use on mobile devices these networks often require too many FLOPs and reducing the FLOPs of a convolution layer, also reduces its parameter count. In this paper we propose to use HyperNetworks to break the fixed ratio of FLOPs to parameters of standard convolutions.
  arXiv  Detail & Related papers  (2021-05-18T12:27:05Z)
• Scalable Visual Transformers with Hierarchical Pooling [61.05787583247392]
  We propose a Hierarchical Visual Transformer (HVT) which progressively pools visual tokens to shrink the sequence length. It brings a great benefit by scaling dimensions of depth/width/resolution/patch size without introducing extra computational complexity. Our HVT outperforms the competitive baselines on ImageNet and CIFAR-100 datasets.
  arXiv  Detail & Related papers  (2021-03-19T03:55:58Z)
• CNNs for JPEGs: A Study in Computational Cost [49.97673761305336]
  Convolutional neural networks (CNNs) have achieved astonishing advances over the past decade. CNNs are capable of learning robust representations of the data directly from the RGB pixels. Deep learning methods capable of learning directly from the compressed domain have been gaining attention in recent years.
  arXiv  Detail & Related papers  (2020-12-26T15:00:10Z)
• Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification [46.885260723836865]
  Deep convolutional neural networks (CNNs) generally improve when fueled with high resolution images. Inspired by the fact that not all regions in an image are task-relevant, we propose a novel framework that performs efficient image classification. Our framework is general and flexible as it is compatible with most of the state-of-the-art light-weighted CNNs.
  arXiv  Detail & Related papers  (2020-10-11T17:55:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.