Portrait Shadow Manipulation
- URL: http://arxiv.org/abs/2005.08925v2
- Date: Wed, 20 May 2020 17:49:55 GMT
- Title: Portrait Shadow Manipulation
- Authors: Xuaner Cecilia Zhang, Jonathan T. Barron, Yun-Ta Tsai, Rohit Pandey,
Xiuming Zhang, Ren Ng, David E. Jacobs
- Abstract summary: Casually-taken portrait photographs often suffer from unflattering lighting and shadowing because of suboptimal conditions in the environment.
We present a computational approach that gives casual photographers some of this control, thereby allowing poorly-lit portraits to be relit post-capture in a realistic and easily-controllable way.
Our approach relies on a pair of neural networks---one to remove foreign shadows cast by external objects, and another to soften facial shadows cast by the features of the subject and to add a synthetic fill light to improve the lighting ratio.
- Score: 37.414681268753526
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Casually-taken portrait photographs often suffer from unflattering lighting
and shadowing because of suboptimal conditions in the environment. Aesthetic
qualities such as the position and softness of shadows and the lighting ratio
between the bright and dark parts of the face are frequently determined by the
constraints of the environment rather than by the photographer. Professionals
address this issue by adding light shaping tools such as scrims, bounce cards,
and flashes. In this paper, we present a computational approach that gives
casual photographers some of this control, thereby allowing poorly-lit
portraits to be relit post-capture in a realistic and easily-controllable way.
Our approach relies on a pair of neural networks---one to remove foreign
shadows cast by external objects, and another to soften facial shadows cast by
the features of the subject and to add a synthetic fill light to improve the
lighting ratio. To train our first network we construct a dataset of real-world
portraits wherein synthetic foreign shadows are rendered onto the face, and we
show that our network learns to remove those unwanted shadows. To train our
second network we use a dataset of Light Stage scans of human subjects to
construct input/output pairs of input images harshly lit by a small light
source, and variably softened and fill-lit output images of each face. We
propose a way to explicitly encode facial symmetry and show that our dataset
and training procedure enable the model to generalize to images taken in the
wild. Together, these networks enable the realistic and aesthetically pleasing
enhancement of shadows and lights in real-world portrait images
Related papers
- Controllable Light Diffusion for Portraits [8.931046902694984]
We introduce light diffusion, a novel method to improve lighting in portraits.
Inspired by professional photographers' diffusers and scrims, our method softens lighting given only a single portrait photo.
arXiv Detail & Related papers (2023-05-08T14:46:28Z) - Learning to Relight Portrait Images via a Virtual Light Stage and
Synthetic-to-Real Adaptation [76.96499178502759]
Relighting aims to re-illuminate the person in the image as if the person appeared in an environment with the target lighting.
Recent methods rely on deep learning to achieve high-quality results.
We propose a new approach that can perform on par with the state-of-the-art (SOTA) relighting methods without requiring a light stage.
arXiv Detail & Related papers (2022-09-21T17:15:58Z) - Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks.
We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
arXiv Detail & Related papers (2022-05-19T06:44:37Z) - Unsupervised Portrait Shadow Removal via Generative Priors [37.46753287881341]
We propose the first unsupervised method for portrait shadow removal without any training data.
Our key idea is to leverage the generative facial priors embedded in the off-the-shelf pretrained StyleGAN2.
Our approach can also be extended to portrait tattoo removal and watermark removal.
arXiv Detail & Related papers (2021-08-07T15:09:36Z) - NeLF: Neural Light-transport Field for Portrait View Synthesis and
Relighting [49.73715814270705]
We use a neural network to predict the light-transport field in 3D space, and from the predicted Neural Light-transport Field (NeLF) produce a portrait from a new camera view under a new environmental lighting.
Our method achieves simultaneous view synthesis and relighting given multi-view portraits as the input, and achieves state-of-the-art results.
arXiv Detail & Related papers (2021-07-26T17:44:52Z) - Self-supervised Outdoor Scene Relighting [92.20785788740407]
We propose a self-supervised approach for relighting.
Our approach is trained only on corpora of images collected from the internet without any user-supervision.
Results show the ability of our technique to produce photo-realistic and physically plausible results, that generalizes to unseen scenes.
arXiv Detail & Related papers (2021-07-07T09:46:19Z) - Towards High Fidelity Face Relighting with Realistic Shadows [21.09340135707926]
Our method learns to predict the ratio (quotient) image between a source image and the target image with the desired lighting.
During training, our model also learns to accurately modify shadows by using estimated shadow masks.
We demonstrate that our proposed method faithfully maintains the local facial details of the subject and can accurately handle hard shadows.
arXiv Detail & Related papers (2021-04-02T00:28:40Z) - Light Stage Super-Resolution: Continuous High-Frequency Relighting [58.09243542908402]
We propose a learning-based solution for the "super-resolution" of scans of human faces taken from a light stage.
Our method aggregates the captured images corresponding to neighboring lights in the stage, and uses a neural network to synthesize a rendering of the face.
Our learned model is able to produce renderings for arbitrary light directions that exhibit realistic shadows and specular highlights.
arXiv Detail & Related papers (2020-10-17T23:40:43Z) - Towards Geometry Guided Neural Relighting with Flash Photography [26.511476565209026]
We propose a framework for image relighting from a single flash photograph with its corresponding depth map using deep learning.
We experimentally validate the advantage of our geometry guided approach over state-of-the-art image-based approaches in intrinsic image decomposition and image relighting.
arXiv Detail & Related papers (2020-08-12T08:03:28Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.