Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild

• URL: http://arxiv.org/abs/2007.15068v1
• Date: Wed, 29 Jul 2020 19:21:02 GMT
• Title: Unselfie: Translating Selfies to Neutral-pose Portraits in the Wild
• Authors: Liqian Ma, Zhe Lin, Connelly Barnes, Alexei A. Efros, Jingwan Lu
• Abstract summary: In selfies, constraints such as human arm length often make the body pose look unnatural. We introduce $textitunselfie$, a novel photographic transformation that automatically translates a selfie into a neutral-pose portrait. We propose a novel nearest pose search module that makes the reposing task easier and enables the generation of multiple neutral-pose results.
• Score: 57.944605468653414
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Due to the ubiquity of smartphones, it is popular to take photos of one's self, or "selfies." Such photos are convenient to take, because they do not require specialized equipment or a third-party photographer. However, in selfies, constraints such as human arm length often make the body pose look unnatural. To address this issue, we introduce $\textit{unselfie}$, a novel photographic transformation that automatically translates a selfie into a neutral-pose portrait. To achieve this, we first collect an unpaired dataset, and introduce a way to synthesize paired training data for self-supervised learning. Then, to $\textit{unselfie}$ a photo, we propose a new three-stage pipeline, where we first find a target neutral pose, inpaint the body texture, and finally refine and composite the person on the background. To obtain a suitable target neutral pose, we propose a novel nearest pose search module that makes the reposing task easier and enables the generation of multiple neutral-pose results among which users can choose the best one they like. Qualitative and quantitative evaluations show the superiority of our pipeline over alternatives.

Related papers

• Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps [30.970209890835793]
  We explore how to create a professional'' version of a person's photograph.<n>A key challenge is to preserve the person's unique identity, face and body features while transforming the photo.<n>Our approach yields high-quality, reposed portraits and achieves strong qualitative and quantitative performance on real-world imagery.
  arXiv  Detail & Related papers  (2025-12-19T00:40:53Z)
• Learning Complex Non-Rigid Image Edits from Multimodal Conditioning [18.500715348636582]
  We focus on inserting a given human (specifically, a single image of a person) into a novel scene. Our method, which builds on top of Stable Diffusion, yields natural looking images while being highly controllable with text and pose. We demonstrate that identity preservation is a more challenging task in scenes "in-the-wild", and especially scenes where there is an interaction between persons and objects.
  arXiv  Detail & Related papers  (2024-12-13T15:41:08Z)
• Learning a Category-level Object Pose Estimator without Pose Annotations [37.03715008347576]
  We propose to learn a category-level 3D object pose estimator without pose annotations. Instead of using manually annotated images, we leverage diffusion models to generate a set of images under controlled pose differences. We show that our method has the capability of category-level object pose estimation from a single shot setting.
  arXiv  Detail & Related papers  (2024-04-08T15:59:29Z)
• MagiCapture: High-Resolution Multi-Concept Portrait Customization [34.131515004434846]
  MagiCapture is a personalization method for integrating subject and style concepts to generate high-resolution portrait images. We present a novel Attention Refocusing loss coupled with auxiliary priors, both of which facilitate robust learning within this weakly supervised learning setting. Our pipeline also includes additional post-processing steps to ensure the creation of highly realistic outputs.
  arXiv  Detail & Related papers  (2023-09-13T11:37:04Z)
• Total Selfie: Generating Full-Body Selfies [21.020454186769655]
  We present a method to generate full-body selfies from photographs originally taken at arms length. Our approach takes as input four selfies of your face and body, a background image, and generates a full-body selfie in a desired target pose.
  arXiv  Detail & Related papers  (2023-08-28T17:41:14Z)
• Fashion-model pose recommendation and generation using Machine Learning [0.0]
  This research concentrates on suggesting the fashion personnel a series of similar images based on the input image. The image is segmented into different parts and similar images are suggested for the user. This was achieved by calculating the color histogram of the input image and applying the same for all the images in the dataset.
  arXiv  Detail & Related papers  (2023-02-19T09:12:46Z)
• TIPS: Text-Induced Pose Synthesis [24.317541784957285]
  In computer vision, human pose synthesis and transfer deal with probabilistic image generation of a person in a previously unseen pose. We first present the shortcomings of current pose transfer algorithms and then propose a novel text-based pose transfer technique to address those issues. The proposed method generates promising results with significant qualitative and quantitative scores in our experiments.
  arXiv  Detail & Related papers  (2022-07-24T11:14:46Z)
• AirPose: Multi-View Fusion Network for Aerial 3D Human Pose and Shape Estimation [51.17610485589701]
  We present a novel markerless 3D human motion capture (MoCap) system for unstructured, outdoor environments. AirPose estimates human pose and shape using images captured by multiple uncalibrated flying cameras. AirPose itself calibrates the cameras relative to the person instead of relying on any pre-calibration.
  arXiv  Detail & Related papers  (2022-01-20T09:46:20Z)
• Estimating Egocentric 3D Human Pose in the Wild with External Weak Supervision [72.36132924512299]
  We present a new egocentric pose estimation method, which can be trained on a large-scale in-the-wild egocentric dataset. We propose a novel learning strategy to supervise the egocentric features with the high-quality features extracted by a pretrained external-view pose estimation model. Experiments show that our method predicts accurate 3D poses from a single in-the-wild egocentric image and outperforms the state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2022-01-20T00:45:13Z)
• Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency [55.94908688207493]
  We propose a self-supervised framework named SPICE that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. SPICE achieves state-of-the-art performance on the DeepFashion dataset.
  arXiv  Detail & Related papers  (2021-10-11T17:48:50Z)
• On Self-Contact and Human Pose [50.96752167102025]
  We develop new datasets and methods that significantly improve human pose estimation with self-contact. We show that the new self-contact training data significantly improves 3D human pose estimates on withheld test data and existing datasets like 3DPW.
  arXiv  Detail & Related papers  (2021-04-07T15:10:38Z)
• FixMyPose: Pose Correctional Captioning and Retrieval [67.20888060019028]
  We introduce a new captioning dataset named FixMyPose to address automated pose correction systems. We collect descriptions of correcting a "current" pose to look like a "target" pose. To avoid ML biases, we maintain a balance across characters with diverse demographics.
  arXiv  Detail & Related papers  (2021-04-04T21:45:44Z)
• PhotoApp: Photorealistic Appearance Editing of Head Portraits [97.23638022484153]
  We present an approach for high-quality intuitive editing of the camera viewpoint and scene illumination in a portrait image. Most editing approaches rely on supervised learning using training data captured with setups such as light and camera stages. We design a supervised problem which learns in the latent space of StyleGAN. This combines the best of supervised learning and generative adversarial modeling.
  arXiv  Detail & Related papers  (2021-03-13T08:59:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.