Normalizing Flows on the Product Space of SO(3) Manifolds for Probabilistic Human Pose Modeling

• URL: http://arxiv.org/abs/2404.05675v1
• Date: Mon, 8 Apr 2024 16:56:05 GMT
• Title: Normalizing Flows on the Product Space of SO(3) Manifolds for Probabilistic Human Pose Modeling
• Authors: Olaf Dünkel, Tim Salzmann, Florian Pfaff,
• Abstract summary: We introduce HuProSO3, a normalizing flow model that operates on a high-dimensional product space of SO(3) manifold. HuProSO3's advantage over state-of-the-art approaches is demonstrated through its superior modeling accuracy in three different applications.
• Score: 4.956709222278244
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Normalizing flows have proven their efficacy for density estimation in Euclidean space, but their application to rotational representations, crucial in various domains such as robotics or human pose modeling, remains underexplored. Probabilistic models of the human pose can benefit from approaches that rigorously consider the rotational nature of human joints. For this purpose, we introduce HuProSO3, a normalizing flow model that operates on a high-dimensional product space of SO(3) manifolds, modeling the joint distribution for human joints with three degrees of freedom. HuProSO3's advantage over state-of-the-art approaches is demonstrated through its superior modeling accuracy in three different applications and its capability to evaluate the exact likelihood. This work not only addresses the technical challenge of learning densities on SO(3) manifolds, but it also has broader implications for domains where the probabilistic regression of correlated 3D rotations is of importance.

Related papers

• Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation [32.30055363306321]
  We propose a paradigm for seamlessly unifying different human pose and shape-related tasks and datasets. Our formulation is centered on the ability - both at training and test time - to query any arbitrary point of the human volume. We can naturally exploit differently annotated data sources including mesh, 2D/3D skeleton and dense pose, without having to convert between them.
  arXiv  Detail & Related papers  (2024-07-10T10:44:18Z)
• $\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation [17.281031933210762]
  We introduce the Discrete Diffusion Pose ($textDi2textPose$), a novel framework designed for occluded 3D human pose estimation. $textDi2textPose$ employs a two-stage process: it first converts 3D poses into a discrete representation through a emphpose quantization step. This methodological innovation restrictively confines the search space towards physically viable configurations.
  arXiv  Detail & Related papers  (2024-05-27T10:01:36Z)
• Unified Generative Modeling of 3D Molecules via Bayesian Flow Networks [19.351562908683334]
  GeoBFN naturally fits molecule geometry by modeling diverse modalities in the differentiable parameter space of distributions. We demonstrate that GeoBFN achieves state-of-the-art performance on multiple 3D molecule generation benchmarks in terms of generation quality. GeoBFN can also conduct sampling with any number of steps to reach an optimal trade-off between efficiency and quality.
  arXiv  Detail & Related papers  (2024-03-17T08:40:06Z)
• Synthetic location trajectory generation using categorical diffusion models [50.809683239937584]
  Diffusion models (DPMs) have rapidly evolved to be one of the predominant generative models for the simulation of synthetic data. We propose using DPMs for the generation of synthetic individual location trajectories (ILTs) which are sequences of variables representing physical locations visited by individuals.
  arXiv  Detail & Related papers  (2024-02-19T15:57:39Z)
• Unified framework for diffusion generative models in SO(3): applications in computer vision and astrophysics [1.3362242492170784]
  Diffusion-based generative models represent the current state-of-the-art for image generation. We develop extensions of both score-based generative models (SGMs) and Denoising Diffusion Probabilistic Models (DDPMs) to the Lie group of 3D rotations, SO(3).
  arXiv  Detail & Related papers  (2023-12-18T21:07:03Z)
• DiffHPE: Robust, Coherent 3D Human Pose Lifting with Diffusion [54.0238087499699]
  We show that diffusion models enhance the accuracy, robustness, and coherence of human pose estimations. We introduce DiffHPE, a novel strategy for harnessing diffusion models in 3D-HPE. Our findings indicate that while standalone diffusion models provide commendable performance, their accuracy is even better in combination with supervised models.
  arXiv  Detail & Related papers  (2023-09-04T12:54:10Z)
• Pose-Oriented Transformer with Uncertainty-Guided Refinement for 2D-to-3D Human Pose Estimation [51.00725889172323]
  We propose a Pose-Oriented Transformer (POT) with uncertainty guided refinement for 3D human pose estimation. We first develop novel pose-oriented self-attention mechanism and distance-related position embedding for POT to explicitly exploit the human skeleton topology. We present an Uncertainty-Guided Refinement Network (UGRN) to refine pose predictions from POT, especially for the difficult joints.
  arXiv  Detail & Related papers  (2023-02-15T00:22:02Z)
• Probabilistic Modeling for Human Mesh Recovery [73.11532990173441]
  This paper focuses on the problem of 3D human reconstruction from 2D evidence. We recast the problem as learning a mapping from the input to a distribution of plausible 3D poses.
  arXiv  Detail & Related papers  (2021-08-26T17:55:11Z)
• THUNDR: Transformer-based 3D HUmaN Reconstruction with Markers [67.8628917474705]
  THUNDR is a transformer-based deep neural network methodology to reconstruct the 3d pose and shape of people. We show state-of-the-art results on Human3.6M and 3DPW, for both the fully-supervised and the self-supervised models. We observe very solid 3d reconstruction performance for difficult human poses collected in the wild.
  arXiv  Detail & Related papers  (2021-06-17T09:09:24Z)
• 3D Multi-bodies: Fitting Sets of Plausible 3D Human Models to Ambiguous Image Data [77.57798334776353]
  We consider the problem of obtaining dense 3D reconstructions of humans from single and partially occluded views. We suggest that ambiguities can be modelled more effectively by parametrizing the possible body shapes and poses. We show that our method outperforms alternative approaches in ambiguous pose recovery on standard benchmarks for 3D humans.
  arXiv  Detail & Related papers  (2020-11-02T13:55:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.