Practical Deep Raw Image Denoising on Mobile Devices

• URL: http://arxiv.org/abs/2010.06935v1
• Date: Wed, 14 Oct 2020 10:30:32 GMT
• Title: Practical Deep Raw Image Denoising on Mobile Devices
• Authors: Yuzhi Wang, Haibin Huang, Qin Xu, Jiaming Liu, Yiqun Liu, Jue Wang
• Abstract summary: We propose a light-weight, efficient neural network-based raw image denoiser that runs smoothly on mainstream mobile devices. Our proposed mobile-friendly denoising model runs at 70 milliseconds per megapixel on Qualcomm Snapdragon 855 chipset.
• Score: 30.3578422624862
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep learning-based image denoising approaches have been extensively studied in recent years, prevailing in many public benchmark datasets. However, the stat-of-the-art networks are computationally too expensive to be directly applied on mobile devices. In this work, we propose a light-weight, efficient neural network-based raw image denoiser that runs smoothly on mainstream mobile devices, and produces high quality denoising results. Our key insights are twofold: (1) by measuring and estimating sensor noise level, a smaller network trained on synthetic sensor-specific data can out-perform larger ones trained on general data; (2) the large noise level variation under different ISO settings can be removed by a novel k-Sigma Transform, allowing a small network to efficiently handle a wide range of noise levels. We conduct extensive experiments to demonstrate the efficiency and accuracy of our approach. Our proposed mobile-friendly denoising model runs at ~70 milliseconds per megapixel on Qualcomm Snapdragon 855 chipset, and it is the basis of the night shot feature of several flagship smartphones released in 2019.

Related papers

• Contrastive and Transfer Learning for Effective Audio Fingerprinting through a Real-World Evaluation Protocol [1.8842532732272859]
  Recent advances in song identification leverage deep neural networks to learn compact audio fingerprints directly from raw waveforms.<n>While these methods perform well under controlled conditions, their accuracy drops significantly in real-world scenarios where the audio is captured via mobile devices in noisy environments.<n>We generate three recordings of the same audio, each with increasing levels of noise, captured using a mobile device's microphone.<n>Our results reveal a substantial performance drop for two state-of-the-art CNN-based models under this protocol, compared to previously reported benchmarks.
  arXiv  Detail & Related papers  (2025-07-08T15:13:26Z)
• Learning from Noise: Enhancing DNNs for Event-Based Vision through Controlled Noise Injection [0.0]
  Event data frequently suffers from considerable noise, negatively impacting the performance and robustness of deep learning models.<n>We propose a novel noise-injection training methodology designed to enhance the robustness against varying levels of event noise.<n>Our approach introduces controlled noise directly into the training data, enabling models to learn noise-resilient representations.
  arXiv  Detail & Related papers  (2025-06-04T13:10:26Z)
• Self-Calibrated Variance-Stabilizing Transformations for Real-World Image Denoising [19.08732222562782]
  Supervised deep learning has become the method of choice for image denoising. We show that, contrary to popular belief, denoising networks specialized in the removal of Gaussian noise can be efficiently leveraged in favor of real-world image denoising.
  arXiv  Detail & Related papers  (2024-07-24T16:23:46Z)
• E-MLB: Multilevel Benchmark for Event-Based Camera Denoising [12.698543500397275]
  Event cameras are more sensitive to junction leakage current and photocurrent as they output differential signals. We construct a large-scale event denoising dataset (multilevel benchmark for event denoising, E-MLB) for the first time. We also propose the first nonreference event denoising metric, the event structural ratio (ESR), which measures the structural intensity of given events.
  arXiv  Detail & Related papers  (2023-03-21T16:31:53Z)
• PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks [115.97113917000145]
  We propose a novel PyNET-V2 Mobile CNN architecture designed specifically for edge devices. The proposed architecture is able to process RAW 12MP photos directly on mobile phones under 1.5 second. We show that the proposed architecture is also compatible with the latest mobile AI accelerators.
  arXiv  Detail & Related papers  (2022-11-08T17:18:01Z)
• Enhancing convolutional neural network generalizability via low-rank weight approximation [6.763245393373041]
  Sufficient denoising is often an important first step for image processing. Deep neural networks (DNNs) have been widely used for image denoising. We introduce a new self-supervised framework for image denoising based on the Tucker low-rank tensor approximation.
  arXiv  Detail & Related papers  (2022-09-26T14:11:05Z)
• Robust Deep Ensemble Method for Real-world Image Denoising [62.099271330458066]
  We propose a simple yet effective Bayesian deep ensemble (BDE) method for real-world image denoising. Our BDE achieves +0.28dB PSNR gain over the state-of-the-art denoising method. Our BDE can be extended to other image restoration tasks, and achieves +0.30dB, +0.18dB and +0.12dB PSNR gains on benchmark datasets.
  arXiv  Detail & Related papers  (2022-06-08T06:19:30Z)
• IDR: Self-Supervised Image Denoising via Iterative Data Refinement [66.5510583957863]
  We present a practical unsupervised image denoising method to achieve state-of-the-art denoising performance. Our method only requires single noisy images and a noise model, which is easily accessible in practical raw image denoising. To evaluate raw image denoising performance in real-world applications, we build a high-quality raw image dataset SenseNoise-500 that contains 500 real-life scenes.
  arXiv  Detail & Related papers  (2021-11-29T07:22:53Z)
• Physics-based Noise Modeling for Extreme Low-light Photography [63.65570751728917]
  We study the noise statistics in the imaging pipeline of CMOS photosensors. We formulate a comprehensive noise model that can accurately characterize the real noise structures. Our noise model can be used to synthesize realistic training data for learning-based low-light denoising algorithms.
  arXiv  Detail & Related papers  (2021-08-04T16:36:29Z)
• Fast Camera Image Denoising on Mobile GPUs with Deep Learning, Mobile AI 2021 Challenge: Report [64.09439666916465]
  We introduce the first Mobile AI challenge, where the target is to develop an end-to-end deep learning-based image denoising solution. The proposed solutions are fully compatible with any mobile GPU and are capable of processing 480p resolution images under 40-80 ms.
  arXiv  Detail & Related papers  (2021-05-17T13:27:56Z)
• Efficient Deep Image Denoising via Class Specific Convolution [24.103826414190216]
  We propose an efficient deep neural network for image denoising based on pixel-wise classification. The proposed method can reduce the computational costs without sacrificing the performance.
  arXiv  Detail & Related papers  (2021-03-02T10:28:15Z)
• Feature-Align Network and Knowledge Distillation for Efficient Denoising [4.997028216419175]
  Deep learning-based RAW image denoising is a quintessential problem in image restoration. We propose a novel network for efficient RAW denoising on mobile devices. We achieve a PSNR of 48.28dB, with 263 times fewer MACs, and 17.6 times fewer parameters than the state-of-the-art network.
  arXiv  Detail & Related papers  (2021-03-02T07:09:32Z)
• Neighbor2Neighbor: Self-Supervised Denoising from Single Noisy Images [98.82804259905478]
  We present Neighbor2Neighbor to train an effective image denoising model with only noisy images. In detail, input and target used to train a network are images sub-sampled from the same noisy image. A denoising network is trained on sub-sampled training pairs generated in the first stage, with a proposed regularizer as additional loss for better performance.
  arXiv  Detail & Related papers  (2021-01-08T02:03:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.