Deep Photo Scan: Semi-supervised learning for dealing with the real-world degradation in smartphone photo scanning

• URL: http://arxiv.org/abs/2102.06120v1
• Date: Thu, 11 Feb 2021 17:00:57 GMT
• Title: Deep Photo Scan: Semi-supervised learning for dealing with the real-world degradation in smartphone photo scanning
• Authors: Man M. Ho, Jinjia Zhou
• Abstract summary: We propose a Deep Photo Scan (DPScan) based on semi-supervised learning. First, we present the way to produce real-world degradation and provide the DIV2K-SCAN dataset for smartphone-scanned photo restoration. Second, by using DIV2K-SCAN, we adopt the concept of Generative Adrial Networks to learn how to degrade a high-quality image as if it were scanned by a real smartphone, then generate pseudo-scanned photos for unscanned photos.
• Score: 12.160561868046122
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Physical photographs now can be conveniently scanned by smartphones and stored forever as a digital version, but the scanned photos are not restored well. One solution is to train a supervised deep neural network on many digital photos and the corresponding scanned photos. However, human annotation costs a huge resource leading to limited training data. Previous works create training pairs by simulating degradation using image processing techniques. Their synthetic images are formed with perfectly scanned photos in latent space. Even so, the real-world degradation in smartphone photo scanning remains unsolved since it is more complicated due to real lens defocus, lighting conditions, losing details via printing, various photo materials, and more. To solve these problems, we propose a Deep Photo Scan (DPScan) based on semi-supervised learning. First, we present the way to produce real-world degradation and provide the DIV2K-SCAN dataset for smartphone-scanned photo restoration. Second, by using DIV2K-SCAN, we adopt the concept of Generative Adversarial Networks to learn how to degrade a high-quality image as if it were scanned by a real smartphone, then generate pseudo-scanned photos for unscanned photos. Finally, we propose to train on the scanned and pseudo-scanned photos representing a semi-supervised approach with a cycle process as: high-quality images --> real-/pseudo-scanned photos --> reconstructed images. The proposed semi-supervised scheme can balance between supervised and unsupervised errors while optimizing to limit imperfect pseudo inputs but still enhance restoration. As a result, the proposed DPScan quantitatively and qualitatively outperforms its baseline architecture, state-of-the-art academic research, and industrial products in smartphone photo scanning.

Related papers

• Descanning: From Scanned to the Original Images with a Color Correction Diffusion Model [11.179584649698134]
  We introduce a new high-quality and large-scale dataset named DESCAN-18K. It contains 18K pairs of original and scanned images collected in the wild containing multiple complex degradations. We propose a new image restoration model called DescanDiffusion consisting of a color encoder that corrects the global color degradation and a conditional denoising diffusion probabilistic model (DDPM) that removes local degradations.
  arXiv  Detail & Related papers  (2024-02-08T02:11:52Z)
• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Perceptual Image Enhancement for Smartphone Real-Time Applications [60.45737626529091]
  We propose LPIENet, a lightweight network for perceptual image enhancement. Our model can deal with noise artifacts, diffraction artifacts, blur, and HDR overexposure. Our model can process 2K resolution images under 1 second in mid-level commercial smartphones.
  arXiv  Detail & Related papers  (2022-10-24T19:16:33Z)
• High Dynamic Range and Super-Resolution from Raw Image Bursts [52.341483902624006]
  This paper introduces the first approach to reconstruct high-resolution, high-dynamic range color images from raw photographic bursts captured by a handheld camera with exposure bracketing. The proposed algorithm is fast, with low memory requirements compared to state-of-the-art learning-based approaches to image restoration. Experiments demonstrate its excellent performance with super-resolution factors of up to $times 4$ on real photographs taken in the wild with hand-held cameras.
  arXiv  Detail & Related papers  (2022-07-29T13:31:28Z)
• Measuring Perceptual Color Differences of Smartphone Photographs [55.9434603885868]
  We put together the largest image dataset for perceptual CD assessment. We make one of the first attempts to construct an end-to-end learnable CD formula based on a lightweight neural network.
  arXiv  Detail & Related papers  (2022-05-26T16:57:04Z)
• Dual-reference Training Data Acquisition and CNN Construction for Image Super-Resolution [33.388234549922025]
  We propose a novel method to capture a large set of realistic LR$sim$HR image pairs using real cameras. Our innovation is to shoot images displayed on an ultra-high quality screen at different resolutions. Experimental results show that training a super-resolution CNN by our LR$sim$HR dataset has superior restoration performance than training it by existing datasets on real world images.
  arXiv  Detail & Related papers  (2021-08-05T03:31:50Z)
• Stereo Matching by Self-supervision of Multiscopic Vision [65.38359887232025]
  We propose a new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera positions. A cross photometric loss, an uncertainty-aware mutual-supervision loss, and a new smoothness loss are introduced to optimize the network. Our model obtains better disparity maps than previous unsupervised methods on the KITTI dataset.
  arXiv  Detail & Related papers  (2021-04-09T02:58:59Z)
• Exploiting Raw Images for Real-Scene Super-Resolution [105.18021110372133]
  We study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. We also develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images.
  arXiv  Detail & Related papers  (2021-02-02T16:10:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.