Depth-aware Blending of Smoothed Images for Bokeh Effect Generation
- URL: http://arxiv.org/abs/2005.14214v1
- Date: Thu, 28 May 2020 18:11:05 GMT
- Title: Depth-aware Blending of Smoothed Images for Bokeh Effect Generation
- Authors: Saikat Dutta
- Abstract summary: In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images.
The network is lightweight and can process an HD image in 0.03 seconds.
This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.
- Score: 10.790210744021072
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Bokeh effect is used in photography to capture images where the closer
objects look sharp and every-thing else stays out-of-focus. Bokeh photos are
generally captured using Single Lens Reflex cameras using shallow
depth-of-field. Most of the modern smartphones can take bokeh images by
leveraging dual rear cameras or a good auto-focus hardware. However, for
smartphones with single-rear camera without a good auto-focus hardware, we have
to rely on software to generate bokeh images. This kind of system is also
useful to generate bokeh effect in already captured images. In this paper, an
end-to-end deep learning framework is proposed to generate high-quality bokeh
effect from images. The original image and different versions of smoothed
images are blended to generate Bokeh effect with the help of a monocular depth
estimation network. The proposed approach is compared against a saliency
detection based baseline and a number of approaches proposed in AIM 2019
Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order
to understand different parts of the proposed algorithm. The network is
lightweight and can process an HD image in 0.03 seconds. This approach ranked
second in AIM 2019 Bokeh effect challenge-Perceptual Track.
Related papers
- Variable Aperture Bokeh Rendering via Customized Focal Plane Guidance [18.390543681127976]
The proposed method has achieved competitive state-of-the-art performance with only 4.4M parameters, which is much lighter than mainstream computational bokeh models.
The proposed method has achieved competitive state-of-the-art performance with only 4.4M parameters, which is much lighter than mainstream computational bokeh models.
arXiv Detail & Related papers (2024-10-18T12:04:23Z) - BokehOrNot: Transforming Bokeh Effect with Image Transformer and Lens
Metadata Embedding [2.3784282912975345]
Bokeh effect is an optical phenomenon that offers a pleasant visual experience, typically generated by high-end cameras with wide aperture lenses.
We propose a novel universal method for embedding lens metadata into the model and introducing a loss calculation method using alpha masks.
Based on the above techniques, we propose the BokehOrNot model, which is capable of producing both blur-to-sharp and sharp-to-blur bokeh effect.
arXiv Detail & Related papers (2023-06-06T21:49:56Z) - Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022
challenge: Report [75.79829464552311]
This challenge was to develop an efficient end-to-end AI-based rendering approach that can run on modern smartphone models.
The resulting model was evaluated on the Kirin 9000's Mali GPU that provides excellent acceleration results for the majority of common deep learning ops.
arXiv Detail & Related papers (2022-11-07T22:42:02Z) - Bokeh-Loss GAN: Multi-Stage Adversarial Training for Realistic
Edge-Aware Bokeh [3.8811606213997587]
We tackle the problem of monocular bokeh synthesis, where we attempt to render a shallow depth of field image from a single all-in-focus image.
Unlike in DSLR cameras, this effect can not be captured directly in mobile cameras due to the physical constraints of the mobile aperture.
We propose a network-based approach that is capable of rendering realistic monocular bokeh from single image inputs.
arXiv Detail & Related papers (2022-08-25T20:57:07Z) - Multi-View Motion Synthesis via Applying Rotated Dual-Pixel Blur Kernels [48.063176079878055]
One of the primary effects applied to images captured in portrait mode is a synthetic shallow depth of field (DoF)
In this work, we follow the trend of rendering the NIMAT effect by introducing a modification on the blur synthesis procedure in portrait mode.
Our modification enables a high-quality synthesis of multi-view bokeh from a single image by applying rotated blurring kernels.
arXiv Detail & Related papers (2021-11-15T15:23:55Z) - Single image deep defocus estimation and its applications [82.93345261434943]
We train a deep neural network to classify image patches into one of the 20 levels of blurriness.
The trained model is used to determine the patch blurriness which is then refined by applying an iterative weighted guided filter.
The result is a defocus map that carries the information of the degree of blurriness for each pixel.
arXiv Detail & Related papers (2021-07-30T06:18:16Z) - AIM 2020 Challenge on Rendering Realistic Bokeh [95.87775182820518]
This paper reviews the second AIM realistic bokeh effect rendering challenge.
The goal was to learn a realistic shallow focus technique using a large-scale EBB! bokeh dataset.
The participants had to render bokeh effect based on only one single frame without any additional data from other cameras or sensors.
arXiv Detail & Related papers (2020-11-10T09:15:38Z) - BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering
Realistic Bokeh [19.752904494597328]
We propose a novel generator called Glass-Net, which generates bokeh images not relying on complex hardware.
Experiments show that our method is able to render a high-quality bokeh effect and process one $1024 times 1536$ pixel image in 1.9 seconds on all smartphone chipsets.
arXiv Detail & Related papers (2020-11-04T11:56:34Z) - Defocus Blur Detection via Depth Distillation [64.78779830554731]
We introduce depth information into DBD for the first time.
In detail, we learn the defocus blur from ground truth and the depth distilled from a well-trained depth estimation network.
Our approach outperforms 11 other state-of-the-art methods on two popular datasets.
arXiv Detail & Related papers (2020-07-16T04:58:09Z) - Rendering Natural Camera Bokeh Effect with Deep Learning [95.86933125733673]
Bokeh is an important artistic effect used to highlight the main object of interest on the photo.
Mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics.
We propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras.
arXiv Detail & Related papers (2020-06-10T07:28:06Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.