Natural & Adversarial Bokeh Rendering via Circle-of-Confusion Predictive Network

• URL: http://arxiv.org/abs/2111.12971v3
• Date: Wed, 29 Nov 2023 14:15:29 GMT
• Title: Natural & Adversarial Bokeh Rendering via Circle-of-Confusion Predictive Network
• Authors: Yihao Huang, Felix Juefei-Xu, Qing Guo, Geguang Pu, Yang Liu
• Abstract summary: Bokeh effect is a shallow depth-of-field phenomenon that blurs out-of-focus part in photography. We study a totally new problem, i.e., natural & adversarial bokeh rendering. We propose a hybrid alternative by taking the respective advantages of data-driven and physical-aware methods.
• Score: 25.319666328268116
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Bokeh effect is a natural shallow depth-of-field phenomenon that blurs the out-of-focus part in photography. In recent years, a series of works have proposed automatic and realistic bokeh rendering methods for artistic and aesthetic purposes. They usually employ cutting-edge data-driven deep generative networks with complex training strategies and network architectures. However, these works neglect that the bokeh effect, as a real phenomenon, can inevitably affect the subsequent visual intelligent tasks like recognition, and their data-driven nature prevents them from studying the influence of bokeh-related physical parameters (i.e., depth-of-the-field) on the intelligent tasks. To fill this gap, we study a totally new problem, i.e., natural & adversarial bokeh rendering, which consists of two objectives: rendering realistic and natural bokeh and fooling the visual perception models (i.e., bokeh-based adversarial attack). To this end, beyond the pure data-driven solution, we propose a hybrid alternative by taking the respective advantages of data-driven and physical-aware methods. Specifically, we propose the circle-of-confusion predictive network (CoCNet) by taking the all-in-focus image and depth image as inputs to estimate circle-of-confusion parameters for each pixel, which are employed to render the final image through a well-known physical model of bokeh. With the hybrid solution, our method could achieve more realistic rendering results with the naive training strategy and a much lighter network.

Related papers

• GBSD: Generative Bokeh with Stage Diffusion [16.189787907983106]
  The bokeh effect is an artistic technique that blurs out-of-focus areas in a photograph. We present GBSD, the first generative text-to-image model that synthesizes photorealistic images with a bokeh style.
  arXiv  Detail & Related papers  (2023-06-14T05:34:02Z)
• Defocus to focus: Photo-realistic bokeh rendering by fusing defocus and radiance priors [26.38833313692807]
  Bokeh rendering mimics aesthetic shallow depth-of-field (DoF) in professional photography. Existing methods suffer from simple flat background blur and blurred in-focus regions. We present a Defocus to Focus (D2F) framework to learn realistic bokeh rendering.
  arXiv  Detail & Related papers  (2023-06-07T15:15:13Z)
• Bokeh-Loss GAN: Multi-Stage Adversarial Training for Realistic Edge-Aware Bokeh [3.8811606213997587]
  We tackle the problem of monocular bokeh synthesis, where we attempt to render a shallow depth of field image from a single all-in-focus image. Unlike in DSLR cameras, this effect can not be captured directly in mobile cameras due to the physical constraints of the mobile aperture. We propose a network-based approach that is capable of rendering realistic monocular bokeh from single image inputs.
  arXiv  Detail & Related papers  (2022-08-25T20:57:07Z)
• Single image deep defocus estimation and its applications [82.93345261434943]
  We train a deep neural network to classify image patches into one of the 20 levels of blurriness. The trained model is used to determine the patch blurriness which is then refined by applying an iterative weighted guided filter. The result is a defocus map that carries the information of the degree of blurriness for each pixel.
  arXiv  Detail & Related papers  (2021-07-30T06:18:16Z)
• BaMBNet: A Blur-aware Multi-branch Network for Defocus Deblurring [74.34263243089688]
  convolutional neural networks (CNNs) have been introduced to the defocus deblurring problem and achieved significant progress. This study designs a novel blur-aware multi-branch network (BaMBNet) in which different regions (with different blur amounts) should be treated differentially. Both quantitative and qualitative experiments demonstrate that our BaMBNet outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-05-31T07:55:30Z)
• AIM 2020 Challenge on Rendering Realistic Bokeh [95.87775182820518]
  This paper reviews the second AIM realistic bokeh effect rendering challenge. The goal was to learn a realistic shallow focus technique using a large-scale EBB! bokeh dataset. The participants had to render bokeh effect based on only one single frame without any additional data from other cameras or sensors.
  arXiv  Detail & Related papers  (2020-11-10T09:15:38Z)
• Defocus Blur Detection via Depth Distillation [64.78779830554731]
  We introduce depth information into DBD for the first time. In detail, we learn the defocus blur from ground truth and the depth distilled from a well-trained depth estimation network. Our approach outperforms 11 other state-of-the-art methods on two popular datasets.
  arXiv  Detail & Related papers  (2020-07-16T04:58:09Z)
• Rendering Natural Camera Bokeh Effect with Deep Learning [95.86933125733673]
  Bokeh is an important artistic effect used to highlight the main object of interest on the photo. Mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics. We propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras.
  arXiv  Detail & Related papers  (2020-06-10T07:28:06Z)
• Depth-aware Blending of Smoothed Images for Bokeh Effect Generation [10.790210744021072]
  In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images. The network is lightweight and can process an HD image in 0.03 seconds. This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.
  arXiv  Detail & Related papers  (2020-05-28T18:11:05Z)
• Self-Supervised Linear Motion Deblurring [112.75317069916579]
  Deep convolutional neural networks are state-of-the-art for image deblurring. We present a differentiable reblur model for self-supervised motion deblurring. Our experiments demonstrate that self-supervised single image deblurring is really feasible.
  arXiv  Detail & Related papers  (2020-02-10T20:15:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.