PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing

• URL: http://arxiv.org/abs/2104.02895v1
• Date: Wed, 7 Apr 2021 03:40:11 GMT
• Title: PyNET-CA: Enhanced PyNET with Channel Attention for End-to-End Mobile Image Signal Processing
• Authors: Byung-Hoon Kim, Joonyoung Song, Jong Chul Ye, JaeHyun Baek
• Abstract summary: We propose PyNET-CA, an end-to-end mobile ISP deep learning algorithm for RAW to RGB reconstruction. We demonstrate the performance of the proposed method with comparative experiments and results from the AIM 2020 learned smartphone ISP challenge.
• Score: 32.7355302269855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Reconstructing RGB image from RAW data obtained with a mobile device is related to a number of image signal processing (ISP) tasks, such as demosaicing, denoising, etc. Deep neural networks have shown promising results over hand-crafted ISP algorithms on solving these tasks separately, or even replacing the whole reconstruction process with one model. Here, we propose PyNET-CA, an end-to-end mobile ISP deep learning algorithm for RAW to RGB reconstruction. The model enhances PyNET, a recently proposed state-of-the-art model for mobile ISP, and improve its performance with channel attention and subpixel reconstruction module. We demonstrate the performance of the proposed method with comparative experiments and results from the AIM 2020 learned smartphone ISP challenge. The source code of our implementation is available at https://github.com/egyptdj/skyb-aim2020-public

Related papers

• Parameter-Inverted Image Pyramid Networks [49.35689698870247]
  We propose a novel network architecture known as the Inverted Image Pyramid Networks (PIIP) Our core idea is to use models with different parameter sizes to process different resolution levels of the image pyramid. PIIP achieves superior performance in tasks such as object detection, segmentation, and image classification.
  arXiv  Detail & Related papers  (2024-06-06T17:59:10Z)
• Simple Image Signal Processing using Global Context Guidance [56.41827271721955]
  Deep learning-based ISPs aim to transform RAW images into DSLR-like RGB images using deep neural networks. We propose a novel module that can be integrated into any neural ISP to capture the global context information from the full RAW images. Our model achieves state-of-the-art results on different benchmarks using diverse and real smartphone images.
  arXiv  Detail & Related papers  (2024-04-17T17:11:47Z)
• Rawformer: Unpaired Raw-to-Raw Translation for Learnable Camera ISPs [53.68932498994655]
  This paper introduces a novel method for unpaired learning of raw-to-raw translation across diverse cameras. It accurately maps raw images captured by a certain camera to the target camera, facilitating the generalization of learnable ISPs to new unseen cameras. Our method demonstrates superior performance on real camera datasets, achieving higher accuracy compared to previous state-of-the-art techniques.
  arXiv  Detail & Related papers  (2024-04-16T16:17:48Z)
• Dual-Scale Transformer for Large-Scale Single-Pixel Imaging [11.064806978728457]
  We propose a deep unfolding network with hybrid-attention Transformer on Kronecker SPI model, dubbed HATNet, to improve the imaging quality of real SPI cameras. The gradient descent module can avoid high computational overheads rooted in previous gradient descent modules based on vectorized SPI. The denoising module is an encoder-decoder architecture powered by dual-scale spatial attention for high- and low-frequency aggregation and channel attention for global information recalibration.
  arXiv  Detail & Related papers  (2024-04-07T15:53:21Z)
• PyNet-V2 Mobile: Efficient On-Device Photo Processing With Neural Networks [115.97113917000145]
  We propose a novel PyNET-V2 Mobile CNN architecture designed specifically for edge devices. The proposed architecture is able to process RAW 12MP photos directly on mobile phones under 1.5 second. We show that the proposed architecture is also compatible with the latest mobile AI accelerators.
  arXiv  Detail & Related papers  (2022-11-08T17:18:01Z)
• LW-ISP: A Lightweight Model with ISP and Deep Learning [17.972611191715888]
  We show the possibility of learning-based method to achieve real-time high-performance processing in the ISP pipeline. We propose LW-ISP, a novel architecture designed to implicitly learn the image mapping from RAW data to RGB image. Experiments demonstrate that LW-ISP has achieved a 0.38 dB improvement in PSNR compared to the previous best method.
  arXiv  Detail & Related papers  (2022-10-08T04:00:03Z)
• Del-Net: A Single-Stage Network for Mobile Camera ISP [14.168130234198467]
  Traditional image signal processing (ISP) pipeline in a smartphone camera consists of several image processing steps performed sequentially to reconstruct a high quality sRGB image from the raw sensor data. Deep learning methods using convolutional neural networks (CNN) have become popular in solving many image-related tasks such as image denoising, contrast enhancement, super resolution, deblurring, etc. In this paper we propose DelNet - a single end-to-end deep learning model - to learn the entire ISP pipeline within reasonable complexity for smartphone deployment.
  arXiv  Detail & Related papers  (2021-08-03T16:51:11Z)
• CNNs for JPEGs: A Study in Computational Cost [49.97673761305336]
  Convolutional neural networks (CNNs) have achieved astonishing advances over the past decade. CNNs are capable of learning robust representations of the data directly from the RGB pixels. Deep learning methods capable of learning directly from the compressed domain have been gaining attention in recent years.
  arXiv  Detail & Related papers  (2020-12-26T15:00:10Z)
• AWNet: Attentive Wavelet Network for Image ISP [14.58067200317891]
  We introduce a novel network that utilizes the attention mechanism and wavelet transform, dubbed AWNet, to tackle this learnable image ISP problem. Our proposed method enables us to restore favorable image details from RAW information and achieve a larger receptive field. Experimental results indicate the advances of our design in both qualitative and quantitative measurements.
  arXiv  Detail & Related papers  (2020-08-20T23:28:41Z)
• Replacing Mobile Camera ISP with a Single Deep Learning Model [171.49776472948957]
  PyNET is a novel pyramidal CNN architecture designed for fine-grained image restoration. The model is trained to convert RAW Bayer data obtained directly from mobile camera sensor into photos captured with a professional high-end DSLR camera.
  arXiv  Detail & Related papers  (2020-02-13T14:22:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.