Sky Optimization: Semantically aware image processing of skies in low-light photography

• URL: http://arxiv.org/abs/2006.10172v1
• Date: Mon, 15 Jun 2020 20:19:12 GMT
• Title: Sky Optimization: Semantically aware image processing of skies in low-light photography
• Authors: Orly Liba, Longqi Cai, Yun-Ta Tsai, Elad Eban, Yair Movshovitz-Attias, Yael Pritch, Huizhong Chen, Jonathan T. Barron
• Abstract summary: We propose an automated method, which can run as a part of a camera pipeline, for creating accurate sky alpha-masks. Our method performs end-to-end sky optimization in less than half a second per image on a mobile device.
• Score: 26.37385679374474
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The sky is a major component of the appearance of a photograph, and its color and tone can strongly influence the mood of a picture. In nighttime photography, the sky can also suffer from noise and color artifacts. For this reason, there is a strong desire to process the sky in isolation from the rest of the scene to achieve an optimal look. In this work, we propose an automated method, which can run as a part of a camera pipeline, for creating accurate sky alpha-masks and using them to improve the appearance of the sky. Our method performs end-to-end sky optimization in less than half a second per image on a mobile device. We introduce a method for creating an accurate sky-mask dataset that is based on partially annotated images that are inpainted and refined by our modified weighted guided filter. We use this dataset to train a neural network for semantic sky segmentation. Due to the compute and power constraints of mobile devices, sky segmentation is performed at a low image resolution. Our modified weighted guided filter is used for edge-aware upsampling to resize the alpha-mask to a higher resolution. With this detailed mask we automatically apply post-processing steps to the sky in isolation, such as automatic spatially varying white-balance, brightness adjustments, contrast enhancement, and noise reduction.

Related papers

• Dimma: Semi-supervised Low Light Image Enhancement with Adaptive Dimming [0.728258471592763]
  Enhancing low-light images while maintaining natural colors is a challenging problem due to camera processing variations. We propose Dimma, a semi-supervised approach that aligns with any camera by utilizing a small set of image pairs. We achieve that by introducing a convolutional mixture density network that generates distorted colors of the scene based on the illumination differences.
  arXiv  Detail & Related papers  (2023-10-14T17:59:46Z)
• ITRE: Low-light Image Enhancement Based on Illumination Transmission Ratio Estimation [10.26197196078661]
  Noise, artifacts, and over-exposure are significant challenges in the field of low-light image enhancement. We propose a novel Retinex-based method, called ITRE, which suppresses noise and artifacts from the origin of the model. Extensive experiments demonstrate the effectiveness of our approach in suppressing noise, preventing artifacts, and controlling over-exposure level simultaneously.
  arXiv  Detail & Related papers  (2023-10-08T13:22:20Z)
• ExBluRF: Efficient Radiance Fields for Extreme Motion Blurred Images [58.24910105459957]
  We present ExBluRF, a novel view synthesis method for extreme motion blurred images. Our approach consists of two main components: 6-DOF camera trajectory-based motion blur formulation and voxel-based radiance fields. Compared with the existing works, our approach restores much sharper 3D scenes with the order of 10 times less training time and GPU memory consumption.
  arXiv  Detail & Related papers  (2023-09-16T11:17:25Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• See Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images [73.23687409870656]
  We propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss.
  arXiv  Detail & Related papers  (2022-10-14T07:45:33Z)
• High Dynamic Range and Super-Resolution from Raw Image Bursts [52.341483902624006]
  This paper introduces the first approach to reconstruct high-resolution, high-dynamic range color images from raw photographic bursts captured by a handheld camera with exposure bracketing. The proposed algorithm is fast, with low memory requirements compared to state-of-the-art learning-based approaches to image restoration. Experiments demonstrate its excellent performance with super-resolution factors of up to $times 4$ on real photographs taken in the wild with hand-held cameras.
  arXiv  Detail & Related papers  (2022-07-29T13:31:28Z)
• Zero-Reference Image Restoration for Under-Display Camera of UAV [10.498049147922258]
  We propose a new method to enhance the visual experience by enhancing the texture and color of images. Our method trains a lightweight network to estimate a low-rank affine grid on the input image. Our model can perform high-quality recovery of images of arbitrary resolution in real time.
  arXiv  Detail & Related papers  (2022-02-13T11:12:00Z)
• Neural Radiance Fields for Outdoor Scene Relighting [70.97747511934705]
  We present NeRF-OSR, the first approach for outdoor scene relighting based on neural radiance fields. In contrast to the prior art, our technique allows simultaneous editing of both scene illumination and camera viewpoint. It also includes a dedicated network for shadow reproduction, which is crucial for high-quality outdoor scene relighting.
  arXiv  Detail & Related papers  (2021-12-09T18:59:56Z)
• Detecting Blurred Ground-based Sky/Cloud Images [5.178207785968322]
  We propose an efficient framework that can identify the blurred sky/cloud images. Using a static external marker, our proposed methodology has a detection accuracy of 94%.
  arXiv  Detail & Related papers  (2021-10-19T06:51:31Z)
• Contextual colorization and denoising for low-light ultra high resolution sequences [0.0]
  Low-light image sequences generally suffer from incoherent noise, flicker and blurring of objects and moving objects. We tackle these problems with an unpaired-learning method that offers simultaneous colorization and denoising. We show that our method outperforms existing approaches in terms of subjective quality and that it is robust to variations in brightness levels and noise.
  arXiv  Detail & Related papers  (2021-01-05T15:35:29Z)
• Deep Photo Cropper and Enhancer [65.11910918427296]
  We propose a new type of image enhancement problem: to crop an image which is embedded within a photo. We split our proposed approach into two deep networks: deep photo cropper and deep image enhancer. In the photo cropper network, we employ a spatial transformer to extract the embedded image. In the photo enhancer, we employ super-resolution to increase the number of pixels in the embedded image.
  arXiv  Detail & Related papers  (2020-08-03T03:50:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.