Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network

• URL: http://arxiv.org/abs/2104.09556v1
• Date: Mon, 19 Apr 2021 18:41:45 GMT
• Title: Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network
• Authors: Ruicheng Feng, Chongyi Li, Huaijin Chen, Shuai Li, Chen Change Loy, Jinwei Gu
• Abstract summary: Under-Display Camera (UDC) systems provide a true bezel-less and notch-free viewing experience on smartphones. In a typical UDC system, the pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. In this work, we aim to analyze and tackle the aforementioned degradation problems.
• Score: 80.67717076541956
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent development of Under-Display Camera (UDC) systems provides a true bezel-less and notch-free viewing experience on smartphones (and TV, laptops, tablets), while allowing images to be captured from the selfie camera embedded underneath. In a typical UDC system, the microstructure of the semi-transparent organic light-emitting diode (OLED) pixel array attenuates and diffracts the incident light on the camera, resulting in significant image quality degradation. Oftentimes, noise, flare, haze, and blur can be observed in UDC images. In this work, we aim to analyze and tackle the aforementioned degradation problems. We define a physics-based image formation model to better understand the degradation. In addition, we utilize one of the world's first commodity UDC smartphone prototypes to measure the real-world Point Spread Function (PSF) of the UDC system, and provide a model-based data synthesis pipeline to generate realistically degraded images. We specially design a new domain knowledge-enabled Dynamic Skip Connection Network (DISCNet) to restore the UDC images. We demonstrate the effectiveness of our method through extensive experiments on both synthetic and real UDC data. Our physics-based image formation model and proposed DISCNet can provide foundations for further exploration in UDC image restoration, and even for general diffraction artifact removal in a broader sense.

Related papers

• Segmentation Guided Sparse Transformer for Under-Display Camera Image Restoration [91.65248635837145]
  Under-Display Camera (UDC) is an emerging technology that achieves full-screen display via hiding the camera under the display panel. In this paper, we observe that when using the Vision Transformer for UDC degraded image restoration, the global attention samples a large amount of redundant information and noise. We propose a Guided Sparse Transformer method (SGSFormer) for the task of restoring high-quality images from UDC degraded images.
  arXiv  Detail & Related papers  (2024-03-09T13:11:59Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• Deep Video Restoration for Under-Display Camera [98.17505013737446]
  We propose a GAN-based generation pipeline to simulate the realistic UDC degradation process. We build the first large-scale UDC video restoration dataset called PexelsUDC. We propose a novel transformer-based baseline method that adaptively enhances degraded videos.
  arXiv  Detail & Related papers  (2023-09-09T10:48:06Z)
• Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
  This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions. The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result. To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
  arXiv  Detail & Related papers  (2023-08-10T07:53:06Z)
• Physics-Driven Turbulence Image Restoration with Stochastic Refinement [80.79900297089176]
  Image distortion by atmospheric turbulence is a critical problem in long-range optical imaging systems. Fast and physics-grounded simulation tools have been introduced to help the deep-learning models adapt to real-world turbulence conditions. This paper proposes the Physics-integrated Restoration Network (PiRN) to help the network to disentangle theity from the degradation and the underlying image.
  arXiv  Detail & Related papers  (2023-07-20T05:49:21Z)
• Modular Degradation Simulation and Restoration for Under-Display Camera [21.048590332029995]
  Under-display camera (UDC) provides an elegant solution for full-screen smartphones. UDC captured images suffer from severe degradation since sensors lie under the display. We propose a modular network dubbed MPGNet trained using the generative adversarial network (GAN) framework for simulating UDC imaging.
  arXiv  Detail & Related papers  (2022-09-23T07:36:07Z)
• UDC-UNet: Under-Display Camera Image Restoration via U-Shape Dynamic Network [13.406025621307132]
  Under-Display Camera (UDC) has been widely exploited to help smartphones realize full screen display. As the screen could inevitably affect the light propagation process, the images captured by the UDC system usually contain flare, haze, blur, and noise. In this paper, we propose a new deep model, namely UDC-UNet, to address the UDC image restoration problem with the known Point Spread Function (PSF) in HDR scenes.
  arXiv  Detail & Related papers  (2022-09-05T07:41:44Z)
• Transform Domain Pyramidal Dilated Convolution Networks For Restoration of Under Display Camera Images [6.731863717520707]
  Under-display camera (UDC) is a novel technology that can make digital imaging experience in handheld devices seamless. UDC images are severely degraded owing to their positioning under a display screen. This work addresses the restoration of images degraded as a result of UDC imaging.
  arXiv  Detail & Related papers  (2020-09-20T09:26:10Z)
• Deep Atrous Guided Filter for Image Restoration in Under Display Cameras [18.6418313982586]
  Under Display Cameras present a promising opportunity for phone manufacturers to achieve bezel-free displays by positioning the camera behind semi-transparent OLED screens. Such imaging systems suffer from severe image degradation due to light attenuation and diffraction effects. We present Deep Atrous Guided Filter (DAGF), a two-stage, end-to-end approach for image restoration in UDC systems.
  arXiv  Detail & Related papers  (2020-08-14T07:54:52Z)
• Image Restoration for Under-Display Camera [14.209602483950322]
  The new trend of full-screen devices encourages us to position a camera behind a screen. Removing the bezel and centralizing the camera under the screen brings larger display-to-body ratio and enhances eye contact in video chat, but also causes image degradation. In this paper, we focus on a newly-defined Under-Display Camera (UDC), as a novel real-world single image restoration problem.
  arXiv  Detail & Related papers  (2020-03-10T17:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.