FlexPose: Pose Distribution Adaptation with Limited Guidance

• URL: http://arxiv.org/abs/2412.13463v1
• Date: Wed, 18 Dec 2024 03:18:11 GMT
• Title: FlexPose: Pose Distribution Adaptation with Limited Guidance
• Authors: Zixiao Wang, Junwu Weng, Mengyuan Liu, Bei Yu,
• Abstract summary: We propose a method to calibrate a pre-trained pose generator in which the pose prior has already been learned to an adapted one following a new pose distribution. We evaluate our proposed method on several cross-dataset settings both qualitatively and quantitatively.
• Score: 15.79919667308626
• License:
• Abstract: Numerous well-annotated human key-point datasets are publicly available to date. However, annotating human poses for newly collected images is still a costly and time-consuming progress. Pose distributions from different datasets share similar pose hinge-structure priors with different geometric transformations, such as pivot orientation, joint rotation, and bone length ratio. The difference between Pose distributions is essentially the difference between the transformation distributions. Inspired by this fact, we propose a method to calibrate a pre-trained pose generator in which the pose prior has already been learned to an adapted one following a new pose distribution. We treat the representation of human pose joint coordinates as skeleton image and transfer a pre-trained pose annotation generator with only a few annotation guidance. By fine-tuning a limited number of linear layers that closely related to the pose transformation, the adapted generator is able to produce any number of pose annotations that are similar to the target poses. We evaluate our proposed method, FlexPose, on several cross-dataset settings both qualitatively and quantitatively, which demonstrates that our approach achieves state-of-the-art performance compared to the existing generative-model-based transfer learning methods when given limited annotation guidance.

Related papers

• GRPose: Learning Graph Relations for Human Image Generation with Pose Priors [21.91374799527015]
  We propose a framework that delves into the graph relations of pose priors to provide control information for human image generation. The main idea is to establish a graph topological structure between the pose priors and latent representation of diffusion models. A pose perception loss is introduced based on a pretrained pose estimation network to minimize the pose differences.
  arXiv  Detail & Related papers  (2024-08-29T13:58:34Z)
• Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation [32.190055780969466]
  Stable-Pose is a novel adapter model that introduces a coarse-to-fine attention masking strategy into a vision Transformer. We leverage the query-key self-attention mechanism of ViTs to explore the interconnections among different anatomical parts in human pose skeletons. Stable-Pose achieved an AP score of 57.1 in the LAION-Human dataset, marking around 13% improvement over the established technique ControlNet.
  arXiv  Detail & Related papers  (2024-06-04T16:54:28Z)
• RePoseDM: Recurrent Pose Alignment and Gradient Guidance for Pose Guided Image Synthesis [14.50214193838818]
  Pose-guided person image synthesis task requires re-rendering a reference image, which should have a photorealistic appearance and flawless pose transfer. We propose recurrent pose alignment to provide pose-aligned texture features as conditional guidance. This helps in learning plausible pose transfer trajectories that result in photorealism and undistorted texture details.
  arXiv  Detail & Related papers  (2023-10-24T15:16:19Z)
• Human Pose as Compositional Tokens [88.28348144244131]
  We present a structured representation, named Pose as Compositional Tokens (PCT), to explore the joint dependency. It represents a pose by M discrete tokens with each characterizing a sub-structure with several interdependent joints. A pre-learned decoder network is used to recover the pose from the tokens without further post-processing.
  arXiv  Detail & Related papers  (2023-03-21T07:14:18Z)
• Open-World Pose Transfer via Sequential Test-Time Adaption [92.67291699304992]
  A typical pose transfer framework usually employs representative datasets to train a discriminative model. Test-time adaption (TTA) offers a feasible solution for OOD data by using a pre-trained model that learns essential features with self-supervision. In our experiment, we first show that pose transfer can be applied to open-world applications, including Tiktok reenactment and celebrity motion synthesis.
  arXiv  Detail & Related papers  (2023-03-20T09:01:23Z)
• PoseTrans: A Simple Yet Effective Pose Transformation Augmentation for Human Pose Estimation [40.50255017107963]
  We propose Pose Transformation (PoseTrans) to create new training samples that have diverse poses. We also propose Pose Clustering Module (PCM) to measure the pose rarity and select the "rarest" poses to help balance the long-tailed distribution. Our method is efficient and simple to implement, which can be easily integrated into the training pipeline of existing pose estimation models.
  arXiv  Detail & Related papers  (2022-08-16T14:03:01Z)
• Few Shot Generative Model Adaption via Relaxed Spatial Structural Alignment [130.84010267004803]
  Training a generative adversarial network (GAN) with limited data has been a challenging task. A feasible solution is to start with a GAN well-trained on a large scale source domain and adapt it to the target domain with a few samples, termed as few shot generative model adaption. We propose a relaxed spatial structural alignment method to calibrate the target generative models during the adaption.
  arXiv  Detail & Related papers  (2022-03-06T14:26:25Z)
• Progressive and Aligned Pose Attention Transfer for Person Image Generation [59.87492938953545]
  This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. We use two types of blocks, namely Pose-Attentional Transfer Block (PATB) and Aligned Pose-Attentional Transfer Bloc (APATB) We verify the efficacy of the model on the Market-1501 and DeepFashion datasets, using quantitative and qualitative measures.
  arXiv  Detail & Related papers  (2021-03-22T07:24:57Z)
• Pose Guided Person Image Generation with Hidden p-Norm Regression [113.41144529452663]
  We propose a novel approach to solve the pose guided person image generation task. Our method estimates a pose-invariant feature matrix for each identity, and uses it to predict the target appearance conditioned on the target pose. Our method yields competitive performance in all the aforementioned variant scenarios.
  arXiv  Detail & Related papers  (2021-02-19T17:03:54Z)
• Adversarial Transfer of Pose Estimation Regression [11.117357750374035]
  We develop a deep adaptation network for learning scene-invariant image representations and use adversarial learning to generate representations for model transfer. We evaluate our network on two public datasets, Cambridge Landmarks and 7Scene, demonstrate its superiority over several baselines and compare to the state of the art methods.
  arXiv  Detail & Related papers  (2020-06-20T21:16:37Z)
• Neural Pose Transfer by Spatially Adaptive Instance Normalization [73.04483812364127]
  We propose the first neural pose transfer model that solves the pose transfer via the latest technique for image style transfer. Our model does not require any correspondences between the source and target meshes. Experiments show that the proposed model can effectively transfer deformation from source to target meshes, and has good generalization ability to deal with unseen identities or poses of meshes.
  arXiv  Detail & Related papers  (2020-03-16T14:33:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.