UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network

• URL: http://arxiv.org/abs/2209.01809v1
• Date: Mon, 5 Sep 2022 07:41:44 GMT
• Title: UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network
• Authors: Xina Liu, Jinfan Hu, Xiangyu Chen, Chao Dong
• Abstract summary: Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. As the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes.
• Score: 13.406025621307132
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. However, as the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. Particularly, flare and blur in UDC images could severely deteriorate the user experience in high dynamic range (HDR) scenes. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes. On the premise that Point Spread Function (PSF) of the UDC system is known, we treat UDC image restoration as a non-blind image restoration problem and propose a novel learning-based approach. Our network consists of three parts, including a U-shape base network to utilize multi-scale information, a condition branch to perform spatially variant modulation, and a kernel branch to provide the prior knowledge of the given PSF. According to the characteristics of HDR data, we additionally design a tone mapping loss to stabilize network optimization and achieve better visual quality. Experimental results show that the proposed UDC-UNet outperforms the state-of-the-art methods in quantitative and qualitative comparisons. Our approach won the second place in the UDC image restoration track of MIPI challenge. Codes will be publicly available.

Related papers

• Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration [91.65248635837145]
  Under-Display Camera (UDC) is an emerging technology that achieves full-screen display via hiding the camera under the display panel. In this paper, we observe that when using the Vision Transformer for UDC degraded image restoration, the global attention samples a large amount of redundant information and noise. We propose a Guided Sparse Transformer method (SGSFormer) for the task of restoring high-quality images from UDC degraded images.
  arXiv  Detail & Related papers  (2024-03-09T13:11:59Z)
• Deep Video Restoration for Under-Display Camera [98.17505013737446]
  We propose a GAN-based generation pipeline to simulate the realistic UDC degradation process. We build the first large-scale UDC video restoration dataset called PexelsUDC. We propose a novel transformer-based baseline method that adaptively enhances degraded videos.
  arXiv  Detail & Related papers  (2023-09-09T10:48:06Z)
• Blind Face Restoration for Under-Display Camera via Dictionary Guided Transformer [32.06570655576273]
  Under-Display Camera (UDC) provides users with a full-screen experience by hiding the front-facing camera below the display panel. UDC images suffer from significant quality degradation due to the characteristics of the display. We propose a two-stage network UDC Degradation Model Network named UDC-DMNet to synthesize UDC images by modeling the processes of UDC imaging.
  arXiv  Detail & Related papers  (2023-08-20T08:02:23Z)
• Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
  This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions. The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result. To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
  arXiv  Detail & Related papers  (2023-08-10T07:53:06Z)
• Generating Aligned Pseudo-Supervision from Non-Aligned Data for Image Restoration in Under-Display Camera [84.41316720913785]
  We revisit the classic stereo setup for training data collection -- capturing two images of the same scene with one UDC and one standard camera. The key idea is to "copy" details from a high-quality reference image and "paste" them on the UDC image. A novel Transformer-based framework generates well-aligned yet high-quality target data for the corresponding UDC input.
  arXiv  Detail & Related papers  (2023-04-12T17:56:42Z)
• Modular Degradation Simulation and Restoration for Under-Display Camera [21.048590332029995]
  Under-display camera (UDC) provides an elegant solution for full-screen smartphones. UDC captured images suffer from severe degradation since sensors lie under the display. We propose a modular network dubbed MPGNet trained using the generative adversarial network (GAN) framework for simulating UDC imaging.
  arXiv  Detail & Related papers  (2022-09-23T07:36:07Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network [80.67717076541956]
  Under-Display Camera (UDC) systems provide a true bezel-less and notch-free viewing experience on smartphones. In a typical UDC system, the pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. In this work, we aim to analyze and tackle the aforementioned degradation problems.
  arXiv  Detail & Related papers  (2021-04-19T18:41:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.