Portrait Shadow Manipulation

• URL: http://arxiv.org/abs/2005.08925v2
• Date: Wed, 20 May 2020 17:49:55 GMT
• Title: Portrait Shadow Manipulation
• Authors: Xuaner Cecilia Zhang, Jonathan T. Barron, Yun-Ta Tsai, Rohit Pandey, Xiuming Zhang, Ren Ng, David E. Jacobs
• Abstract summary: Casually-taken portrait photographs often suffer from unflattering lighting and shadowing because of suboptimal conditions in the environment. We present a computational approach that gives casual photographers some of this control, thereby allowing poorly-lit portraits to be relit post-capture in a realistic and easily-controllable way. Our approach relies on a pair of neural networks---one to remove foreign shadows cast by external objects, and another to soften facial shadows cast by the features of the subject and to add a synthetic fill light to improve the lighting ratio.
• Score: 37.414681268753526
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Casually-taken portrait photographs often suffer from unflattering lighting and shadowing because of suboptimal conditions in the environment. Aesthetic qualities such as the position and softness of shadows and the lighting ratio between the bright and dark parts of the face are frequently determined by the constraints of the environment rather than by the photographer. Professionals address this issue by adding light shaping tools such as scrims, bounce cards, and flashes. In this paper, we present a computational approach that gives casual photographers some of this control, thereby allowing poorly-lit portraits to be relit post-capture in a realistic and easily-controllable way. Our approach relies on a pair of neural networks---one to remove foreign shadows cast by external objects, and another to soften facial shadows cast by the features of the subject and to add a synthetic fill light to improve the lighting ratio. To train our first network we construct a dataset of real-world portraits wherein synthetic foreign shadows are rendered onto the face, and we show that our network learns to remove those unwanted shadows. To train our second network we use a dataset of Light Stage scans of human subjects to construct input/output pairs of input images harshly lit by a small light source, and variably softened and fill-lit output images of each face. We propose a way to explicitly encode facial symmetry and show that our dataset and training procedure enable the model to generalize to images taken in the wild. Together, these networks enable the realistic and aesthetically pleasing enhancement of shadows and lights in real-world portrait images

Related papers

• All-frequency Full-body Human Image Relighting [1.529342790344802]
  Relighting of human images enables post-photography editing of lighting effects in portraits. The current mainstream approach uses neural networks to approximate lighting effects without explicitly accounting for the principle of physical shading. We propose a two-stage relighting method that can reproduce physically-based shadows and shading from low to high frequencies.
  arXiv  Detail & Related papers  (2024-11-01T04:45:48Z)
• Generative Portrait Shadow Removal [27.98144439007323]
  We introduce a high-fidelity portrait shadow removal model that can effectively enhance the image of a portrait. Our method also demonstrates robustness to diverse subjects captured in real environments.
  arXiv  Detail & Related papers  (2024-10-07T22:09:22Z)
• COMPOSE: Comprehensive Portrait Shadow Editing [25.727386174616868]
  COMPOSE is a novel shadow editing pipeline for human portraits. It offers precise control over shadow attributes such as shape, intensity, and position. We have trained models to: (1) predict this light source representation from images, and (2) generate realistic shadows using this representation.
  arXiv  Detail & Related papers  (2024-08-25T19:18:18Z)
• Controllable Light Diffusion for Portraits [8.931046902694984]
  We introduce light diffusion, a novel method to improve lighting in portraits. Inspired by professional photographers' diffusers and scrims, our method softens lighting given only a single portrait photo.
  arXiv  Detail & Related papers  (2023-05-08T14:46:28Z)
• Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation [76.96499178502759]
  Relighting aims to re-illuminate the person in the image as if the person appeared in an environment with the target lighting. Recent methods rely on deep learning to achieve high-quality results. We propose a new approach that can perform on par with the state-of-the-art (SOTA) relighting methods without requiring a light stage.
  arXiv  Detail & Related papers  (2022-09-21T17:15:58Z)
• Physically-Based Editing of Indoor Scene Lighting from a Single Image [106.60252793395104]
  We present a method to edit complex indoor lighting from a single image with its predicted depth and light source segmentation masks. We tackle this problem using two novel components: 1) a holistic scene reconstruction method that estimates scene reflectance and parametric 3D lighting, and 2) a neural rendering framework that re-renders the scene from our predictions.
  arXiv  Detail & Related papers  (2022-05-19T06:44:37Z)
• NeLF: Neural Light-transport Field for Portrait View Synthesis and Relighting [49.73715814270705]
  We use a neural network to predict the light-transport field in 3D space, and from the predicted Neural Light-transport Field (NeLF) produce a portrait from a new camera view under a new environmental lighting. Our method achieves simultaneous view synthesis and relighting given multi-view portraits as the input, and achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2021-07-26T17:44:52Z)
• Self-supervised Outdoor Scene Relighting [92.20785788740407]
  We propose a self-supervised approach for relighting. Our approach is trained only on corpora of images collected from the internet without any user-supervision. Results show the ability of our technique to produce photo-realistic and physically plausible results, that generalizes to unseen scenes.
  arXiv  Detail & Related papers  (2021-07-07T09:46:19Z)
• Towards High Fidelity Face Relighting with Realistic Shadows [21.09340135707926]
  Our method learns to predict the ratio (quotient) image between a source image and the target image with the desired lighting. During training, our model also learns to accurately modify shadows by using estimated shadow masks. We demonstrate that our proposed method faithfully maintains the local facial details of the subject and can accurately handle hard shadows.
  arXiv  Detail & Related papers  (2021-04-02T00:28:40Z)
• Light Stage Super-Resolution: Continuous High-Frequency Relighting [58.09243542908402]
  We propose a learning-based solution for the "super-resolution" of scans of human faces taken from a light stage. Our method aggregates the captured images corresponding to neighboring lights in the stage, and uses a neural network to synthesize a rendering of the face. Our learned model is able to produce renderings for arbitrary light directions that exhibit realistic shadows and specular highlights.
  arXiv  Detail & Related papers  (2020-10-17T23:40:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.