Photo2Relief: Let Human in the Photograph Stand Out

• URL: http://arxiv.org/abs/2307.11364v1
• Date: Fri, 21 Jul 2023 05:33:57 GMT
• Title: Photo2Relief: Let Human in the Photograph Stand Out
• Authors: Zhongping Ji, Feifei Che, Hanshuo Liu, Ziyi Zhao, Yu-Wei Zhang and Wenping Wang
• Abstract summary: We introduce a sigmoid variant function to manipulate gradients tactfully and train our neural networks by equipping with a loss function defined in gradient domain. To make a clear division of labor in network modules, a two-scale architecture is proposed to create high-quality relief from a single photograph.
• Score: 26.102307166656157
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we propose a technique for making humans in photographs protrude like reliefs. Unlike previous methods which mostly focus on the face and head, our method aims to generate art works that describe the whole body activity of the character. One challenge is that there is no ground-truth for supervised deep learning. We introduce a sigmoid variant function to manipulate gradients tactfully and train our neural networks by equipping with a loss function defined in gradient domain. The second challenge is that actual photographs often across different light conditions. We used image-based rendering technique to address this challenge and acquire rendering images and depth data under different lighting conditions. To make a clear division of labor in network modules, a two-scale architecture is proposed to create high-quality relief from a single photograph. Extensive experimental results on a variety of scenes show that our method is a highly effective solution for generating digital 2.5D artwork from photographs.

Related papers

• MoLE: Enhancing Human-centric Text-to-image Diffusion via Mixture of Low-rank Experts [61.274246025372044]
  We study human-centric text-to-image generation in context of faces and hands. We propose a method called Mixture of Low-rank Experts (MoLE) by considering low-rank modules trained on close-up hand and face images respectively as experts. This concept draws inspiration from our observation of low-rank refinement, where a low-rank module trained by a customized close-up dataset has the potential to enhance the corresponding image part when applied at an appropriate scale.
  arXiv  Detail & Related papers  (2024-10-30T17:59:57Z)
• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation [76.96499178502759]
  Relighting aims to re-illuminate the person in the image as if the person appeared in an environment with the target lighting. Recent methods rely on deep learning to achieve high-quality results. We propose a new approach that can perform on par with the state-of-the-art (SOTA) relighting methods without requiring a light stage.
  arXiv  Detail & Related papers  (2022-09-21T17:15:58Z)
• Time-of-Day Neural Style Transfer for Architectural Photographs [18.796803920214238]
  We focus on a neural style transfer method for architectural photography. Our method addresses the composition of the foreground and background in an architectural photograph. Our experiments show that our method can produce photorealistic lighting and color rendition on both the foreground and background.
  arXiv  Detail & Related papers  (2022-09-13T08:00:33Z)
• Explicitly Controllable 3D-Aware Portrait Generation [42.30481422714532]
  We propose a 3D portrait generation network that produces consistent portraits according to semantic parameters regarding pose, identity, expression and lighting. Our method outperforms prior arts in extensive experiments, producing realistic portraits with vivid expression in natural lighting when viewed in free viewpoint.
  arXiv  Detail & Related papers  (2022-09-12T17:40:08Z)
• AvatarMe++: Facial Shape and BRDF Inference with Photorealistic Rendering-Aware GANs [119.23922747230193]
  We introduce the first method that is able to reconstruct render-ready 3D facial geometry and BRDF from a single "in-the-wild" image. Our method outperforms the existing arts by a significant margin and reconstructs high-resolution 3D faces from a single low-resolution image.
  arXiv  Detail & Related papers  (2021-12-11T11:36:30Z)
• Non-Homogeneous Haze Removal via Artificial Scene Prior and Bidimensional Graph Reasoning [52.07698484363237]
  We propose a Non-Homogeneous Haze Removal Network (NHRN) via artificial scene prior and bidimensional graph reasoning. Our method achieves superior performance over many state-of-the-art algorithms for both the single image dehazing and hazy image understanding tasks.
  arXiv  Detail & Related papers  (2021-04-05T13:04:44Z)
• Self-Adaptively Learning to Demoire from Focused and Defocused Image Pairs [97.67638106818613]
  Moire artifacts are common in digital photography, resulting from the interference between high-frequency scene content and the color filter array of the camera. Existing deep learning-based demoireing methods trained on large scale iteration are limited in handling various complex moire patterns. We propose a self-adaptive learning method for demoireing a high-frequency image, with the help of an additional defocused moire-free blur image.
  arXiv  Detail & Related papers  (2020-11-03T23:09:02Z)
• High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images [10.03187850132035]
  We propose an algorithm that matches a non-photorealistic, synthetically generated image to a latent vector of a pretrained StyleGAN2 model. In contrast to most previous work, we require no synthetic training data. This is the first algorithm of its kind to work at a resolution of 1K and represents a significant leap forward in visual realism.
  arXiv  Detail & Related papers  (2020-06-26T15:00:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.