Pose Guided Person Image Generation with Hidden p-Norm Regression
- URL: http://arxiv.org/abs/2102.10033v1
- Date: Fri, 19 Feb 2021 17:03:54 GMT
- Title: Pose Guided Person Image Generation with Hidden p-Norm Regression
- Authors: Ting-Yao Hu, Alexander G. Hauptmann
- Abstract summary: We propose a novel approach to solve the pose guided person image generation task.
Our method estimates a pose-invariant feature matrix for each identity, and uses it to predict the target appearance conditioned on the target pose.
Our method yields competitive performance in all the aforementioned variant scenarios.
- Score: 113.41144529452663
- License: http://creativecommons.org/licenses/by-nc-sa/4.0/
- Abstract: In this paper, we propose a novel approach to solve the pose guided person
image generation task. We assume that the relation between pose and appearance
information can be described by a simple matrix operation in hidden space.
Based on this assumption, our method estimates a pose-invariant feature matrix
for each identity, and uses it to predict the target appearance conditioned on
the target pose. The estimation process is formulated as a p-norm regression
problem in hidden space. By utilizing the differentiation of the solution of
this regression problem, the parameters of the whole framework can be trained
in an end-to-end manner. While most previous works are only applicable to the
supervised training and single-shot generation scenario, our method can be
easily adapted to unsupervised training and multi-shot generation. Extensive
experiments on the challenging Market-1501 dataset show that our method yields
competitive performance in all the aforementioned variant scenarios.
Related papers
- EnsIR: An Ensemble Algorithm for Image Restoration via Gaussian Mixture Models [70.60381055741391]
Image restoration challenges related to illposed problems, resulting in deviations between single model predictions and ground-truths.
Ensemble learning aims to address these deviations by combining the predictions of multiple base models.
We employ an expectation (EM)-based algorithm to estimate ensemble weights for prediction candidates.
Our algorithm is model-agnostic and training-free, allowing seamless integration and enhancement of various pre-trained image restoration models.
arXiv Detail & Related papers (2024-10-30T12:16:35Z) - Conditional Variational Autoencoders for Probabilistic Pose Regression [45.563533339332615]
We propose a probabilistic method to predict the posterior distribution of camera poses given an observed image.
Our proposed training strategy results in a generative model of camera poses given an image, which can be used to draw samples from the pose posterior distribution.
arXiv Detail & Related papers (2024-10-07T12:43:50Z) - Meta-Prior: Meta learning for Adaptive Inverse Problem Solvers [9.364509804053275]
Real-world imaging challenges often lack ground truth data, rendering traditional supervised approaches ineffective.
Our method trains a meta-model on a diverse set of imaging tasks that allows the model to be efficiently fine-tuned for specific tasks.
In simple settings, this approach recovers the Bayes optimal estimator, illustrating the soundness of our approach.
arXiv Detail & Related papers (2023-11-30T17:02:27Z) - Towards Robust and Unconstrained Full Range of Rotation Head Pose
Estimation [2.915868985330569]
We present a novel method for unconstrained end-to-end head pose estimation.
We propose a continuous 6D rotation matrix representation for efficient and robust direct regression.
Our method significantly outperforms other state-of-the-art methods in an efficient and robust manner.
arXiv Detail & Related papers (2023-09-14T12:17:38Z) - Weakly-supervised 3D Pose Transfer with Keypoints [57.66991032263699]
Main challenges of 3D pose transfer are: 1) Lack of paired training data with different characters performing the same pose; 2) Disentangling pose and shape information from the target mesh; 3) Difficulty in applying to meshes with different topologies.
We propose a novel weakly-supervised keypoint-based framework to overcome these difficulties.
arXiv Detail & Related papers (2023-07-25T12:40:24Z) - UMat: Uncertainty-Aware Single Image High Resolution Material Capture [2.416160525187799]
We propose a learning-based method to recover normals, specularity, and roughness from a single diffuse image of a material.
Our method is the first one to deal with the problem of modeling uncertainty in material digitization.
arXiv Detail & Related papers (2023-05-25T17:59:04Z) - PoseMatcher: One-shot 6D Object Pose Estimation by Deep Feature Matching [51.142988196855484]
We propose PoseMatcher, an accurate model free one-shot object pose estimator.
We create a new training pipeline for object to image matching based on a three-view system.
To enable PoseMatcher to attend to distinct input modalities, an image and a pointcloud, we introduce IO-Layer.
arXiv Detail & Related papers (2023-04-03T21:14:59Z) - Poseur: Direct Human Pose Regression with Transformers [119.79232258661995]
We propose a direct, regression-based approach to 2D human pose estimation from single images.
Our framework is end-to-end differentiable, and naturally learns to exploit the dependencies between keypoints.
Ours is the first regression-based approach to perform favorably compared to the best heatmap-based pose estimation methods.
arXiv Detail & Related papers (2022-01-19T04:31:57Z) - Reassessing the Limitations of CNN Methods for Camera Pose Regression [27.86655424544118]
We propose a model that can directly regress the camera pose from images with significantly higher accuracy than existing methods of the same class.
We first analyse why regression methods are still behind the state-of-the-art, and we bridge the performance gap with our new approach.
arXiv Detail & Related papers (2021-08-16T17:55:26Z) - TFPose: Direct Human Pose Estimation with Transformers [83.03424247905869]
We formulate the pose estimation task into a sequence prediction problem that can effectively be solved by transformers.
Our framework is simple and direct, bypassing the drawbacks of the heatmap-based pose estimation.
Experiments on the MS-COCO and MPII datasets demonstrate that our method can significantly improve the state-of-the-art of regression-based pose estimation.
arXiv Detail & Related papers (2021-03-29T04:18:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.