PoseSyn: Synthesizing Diverse 3D Pose Data from In-the-Wild 2D Data

• URL: http://arxiv.org/abs/2503.13025v1
• Date: Mon, 17 Mar 2025 10:28:35 GMT
• Title: PoseSyn: Synthesizing Diverse 3D Pose Data from In-the-Wild 2D Data
• Authors: ChangHee Yang, Hyeonseop Song, Seokhun Choi, Seungwoo Lee, Jaechul Kim, Hoseok Do,
• Abstract summary: PoseSyn is a novel data synthesis framework that transforms abundant in the wild 2D pose dataset into diverse 3D pose image pairs.<n>By generating realistic 3D training data via a human animation model aligned with challenging poses and appearances PoseSyn boosts the accuracy of various 3D pose estimators by up to 14%.
• Score: 1.264462543503282
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite considerable efforts to enhance the generalization of 3D pose estimators without costly 3D annotations, existing data augmentation methods struggle in real world scenarios with diverse human appearances and complex poses. We propose PoseSyn, a novel data synthesis framework that transforms abundant in the wild 2D pose dataset into diverse 3D pose image pairs. PoseSyn comprises two key components: Error Extraction Module (EEM), which identifies challenging poses from the 2D pose datasets, and Motion Synthesis Module (MSM), which synthesizes motion sequences around the challenging poses. Then, by generating realistic 3D training data via a human animation model aligned with challenging poses and appearances PoseSyn boosts the accuracy of various 3D pose estimators by up to 14% across real world benchmarks including various backgrounds and occlusions, challenging poses, and multi view scenarios. Extensive experiments further confirm that PoseSyn is a scalable and effective approach for improving generalization without relying on expensive 3D annotations, regardless of the pose estimator's model size or design.

Related papers

• Mocap-2-to-3: Lifting 2D Diffusion-Based Pretrained Models for 3D Motion Capture [31.82852393452607]
  Mocap-2-to-3 is a novel framework that decomposes intricate 3D motions into 2D poses.<n>We leverage 2D data to enhance 3D motion reconstruction in diverse scenarios.<n>We evaluate our model's performance on real-world datasets.
  arXiv  Detail & Related papers  (2025-03-05T06:32:49Z)
• Lifting Motion to the 3D World via 2D Diffusion [19.64801640086107]
  We introduce MVLift, a novel approach to predict global 3D motion using only 2D pose sequences for training.<n> MVLift generalizes across various domains, including human poses, human-object interactions, and animal poses.
  arXiv  Detail & Related papers  (2024-11-27T23:26:56Z)
• MPL: Lifting 3D Human Pose from Multi-view 2D Poses [75.26416079541723]
  We propose combining 2D pose estimation, for which large and rich training datasets exist, and 2D-to-3D pose lifting, using a transformer-based network. Our experiments demonstrate decreases up to 45% in MPJPE errors compared to the 3D pose obtained by triangulating the 2D poses.
  arXiv  Detail & Related papers  (2024-08-20T12:55:14Z)
• UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues [55.69339788566899]
  UPose3D is a novel approach for multi-view 3D human pose estimation. It improves robustness and flexibility without requiring direct 3D annotations.
  arXiv  Detail & Related papers  (2024-04-23T00:18:00Z)
• Two Views Are Better than One: Monocular 3D Pose Estimation with Multiview Consistency [0.493599216374976]
  We propose a novel loss function, multiview consistency, to enable adding additional training data with only 2D supervision. Our experiments demonstrate that two views offset by 90 degrees are enough to obtain good performance, with only marginal improvements by adding more views. This research introduces new possibilities for domain adaptation in 3D pose estimation, providing a practical and cost-effective solution to customize models for specific applications.
  arXiv  Detail & Related papers  (2023-11-21T08:21:55Z)
• MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling [59.74064212110042]
  mpmcan handle multiple tasks including 3D human pose estimation, 3D pose estimation from cluded 2D pose, and 3D pose completion in a textocbfsingle framework. We conduct extensive experiments and ablation studies on several widely used human pose datasets and achieve state-of-the-art performance on MPI-INF-3DHP.
  arXiv  Detail & Related papers  (2023-06-29T10:30:00Z)
• Decanus to Legatus: Synthetic training for 2D-3D human pose lifting [26.108023246654646]
  We propose an algorithm to generate infinite 3D synthetic human poses (Legatus) from a 3D pose distribution based on 10 initial handcrafted 3D poses (Decanus) Our results show that we can achieve 3D pose estimation performance comparable to methods using real data from specialized datasets but in a zero-shot setup, showing the potential of our framework.
  arXiv  Detail & Related papers  (2022-10-05T13:10:19Z)
• PoseAug: A Differentiable Pose Augmentation Framework for 3D Human Pose Estimation [83.50127973254538]
  Existing 3D human pose estimators suffer poor generalization performance to new datasets. We present PoseAug, a new auto-augmentation framework that learns to augment the available training poses towards a greater diversity.
  arXiv  Detail & Related papers  (2021-05-06T06:57:42Z)
• Self-Supervised 3D Human Pose Estimation via Part Guided Novel Image Synthesis [72.34794624243281]
  We propose a self-supervised learning framework to disentangle variations from unlabeled video frames. Our differentiable formalization, bridging the representation gap between the 3D pose and spatial part maps, allows us to operate on videos with diverse camera movements.
  arXiv  Detail & Related papers  (2020-04-09T07:55:01Z)
• Exemplar Fine-Tuning for 3D Human Model Fitting Towards In-the-Wild 3D Human Pose Estimation [107.07047303858664]
  Large-scale human datasets with 3D ground-truth annotations are difficult to obtain in the wild. We address this problem by augmenting existing 2D datasets with high-quality 3D pose fits. The resulting annotations are sufficient to train from scratch 3D pose regressor networks that outperform the current state-of-the-art on in-the-wild benchmarks.
  arXiv  Detail & Related papers  (2020-04-07T20:21:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.