Wish You Were Here: Context-Aware Human Generation

• URL: http://arxiv.org/abs/2005.10663v1
• Date: Thu, 21 May 2020 14:09:14 GMT
• Title: Wish You Were Here: Context-Aware Human Generation
• Authors: Oran Gafni, Lior Wolf
• Abstract summary: We present a novel method for inserting objects, specifically humans, into existing images. Our method involves threeworks: the first generates the semantic map of the new person, given the pose of the other persons in the scene. The second network renders the pixels of the novel person and its blending mask, based on specifications in the form of multiple appearance components. A third network refines the generated face in order to match those of the target person.
• Score: 100.51309746913512
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present a novel method for inserting objects, specifically humans, into existing images, such that they blend in a photorealistic manner, while respecting the semantic context of the scene. Our method involves three subnetworks: the first generates the semantic map of the new person, given the pose of the other persons in the scene and an optional bounding box specification. The second network renders the pixels of the novel person and its blending mask, based on specifications in the form of multiple appearance components. A third network refines the generated face in order to match those of the target person. Our experiments present convincing high-resolution outputs in this novel and challenging application domain. In addition, the three networks are evaluated individually, demonstrating for example, state of the art results in pose transfer benchmarks.

Related papers

• Single Image, Any Face: Generalisable 3D Face Generation [59.9369171926757]
  We propose a novel model, Gen3D-Face, which generates 3D human faces with unconstrained single image input. To the best of our knowledge, this is the first attempt and benchmark for creating photorealistic 3D human face avatars from single images.
  arXiv  Detail & Related papers  (2024-09-25T14:56:37Z)
• Text2Place: Affordance-aware Text Guided Human Placement [26.041917073228483]
  This work tackles the problem of realistic human insertion in a given background scene termed as textbfSemantic Human Placement. For learning semantic masks, we leverage rich object-scene priors learned from the text-to-image generative models. The proposed method can generate highly realistic scene compositions while preserving the background and subject identity.
  arXiv  Detail & Related papers  (2024-07-22T08:00:06Z)
• HR Human: Modeling Human Avatars with Triangular Mesh and High-Resolution Textures from Videos [52.23323966700072]
  We present a framework for acquiring human avatars that are attached with high-resolution physically-based material textures and mesh from monocular video. Our method introduces a novel information fusion strategy to combine the information from the monocular video and synthesize virtual multi-view images. Experiments show that our approach outperforms previous representations in terms of high fidelity, and this explicit result supports deployment on common triangulars.
  arXiv  Detail & Related papers  (2024-05-18T11:49:09Z)
• Global Context-Aware Person Image Generation [24.317541784957285]
  We propose a data-driven approach for context-aware person image generation. In our method, the position, scale, and appearance of the generated person are semantically conditioned on the existing persons in the scene.
  arXiv  Detail & Related papers  (2023-02-28T16:34:55Z)
• UMFuse: Unified Multi View Fusion for Human Editing applications [36.94334399493266]
  We design a multi-view fusion network that takes the pose key points and texture from multiple source images. We show the application of our network on two newly proposed tasks - Multi-view human reposing and Mix&Match Human Image generation.
  arXiv  Detail & Related papers  (2022-11-17T05:09:58Z)
• Scene Aware Person Image Generation through Global Contextual Conditioning [24.317541784957285]
  We propose a novel pipeline to generate and insert contextually relevant person images into an existing scene. More specifically, we aim to insert a person such that the location, pose, and scale of the person being inserted blends in with the existing persons in the scene.
  arXiv  Detail & Related papers  (2022-06-06T16:18:15Z)
• NeuralReshaper: Single-image Human-body Retouching with Deep Neural Networks [50.40798258968408]
  We present NeuralReshaper, a novel method for semantic reshaping of human bodies in single images using deep generative networks. Our approach follows a fit-then-reshape pipeline, which first fits a parametric 3D human model to a source human image. To deal with the lack-of-data problem that no paired data exist, we introduce a novel self-supervised strategy to train our network.
  arXiv  Detail & Related papers  (2022-03-20T09:02:13Z)
• Stylizing 3D Scene via Implicit Representation and HyperNetwork [34.22448260525455]
  A straightforward solution is to combine existing novel view synthesis and image/video style transfer approaches. Inspired by the high quality results of the neural radiance fields (NeRF) method, we propose a joint framework to directly render novel views with the desired style. Our framework consists of two components: an implicit representation of the 3D scene with the neural radiance field model, and a hypernetwork to transfer the style information into the scene representation.
  arXiv  Detail & Related papers  (2021-05-27T09:11:30Z)
• Learned Spatial Representations for Few-shot Talking-Head Synthesis [68.3787368024951]
  We propose a novel approach for few-shot talking-head synthesis. We show that this disentangled representation leads to a significant improvement over previous methods.
  arXiv  Detail & Related papers  (2021-04-29T17:59:42Z)
• Holistic 3D Scene Understanding from a Single Image with Implicit Representation [112.40630836979273]
  We present a new pipeline for holistic 3D scene understanding from a single image. We propose an image-based local structured implicit network to improve the object shape estimation. We also refine 3D object pose and scene layout via a novel implicit scene graph neural network.
  arXiv  Detail & Related papers  (2021-03-11T02:52:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.