An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectories

• URL: http://arxiv.org/abs/2605.09231v3
• Date: Fri, 15 May 2026 13:23:17 GMT
• Title: An Elastic Shape Variational Autoencoder for Skeleton Pose Trajectories
• Abstract summary: Elastic Shape - Variational Autoencoder (ES-VAE) is a geometry-aware generative model for skeletal trajectories.<n>We demonstrate the effectiveness of ES-VAE on two datasets.
• Score: 8.678193782239054
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Deep generative models provide flexible frameworks for modeling complex, structured data such as images, videos, 3D objects, and texts. However, when applied to sequences of human skeletons, standard variational autoencoders (VAEs) often allocate substantial capacity to nuisance factors-such as camera orientation, subject scale, viewpoint, and execution speed-rather than the intrinsic geometry of shapes and their motion. We propose the Elastic Shape - Variational Autoencoder (ES-VAE), a geometry-aware generative model for skeletal trajectories that leverages the transported square-root velocity field (TSRVF) representation on Kendall's shape manifold. This representation inherently removes rigid translations, rotations, and global scaling of shapes, and temporal rate variability of sequences, isolating the underlying shape dynamics. The ES-VAE encoder maps skeletal sequences to a low-dimensional latent space incorporating the Riemannian logarithm map, while the decoder reconstructs sequences using the corresponding exponential map. We demonstrate the effectiveness of ES-VAE on two datasets. First, we analyze skeletal gait cycles to predict clinical mobility scores and classify subjects into healthy and post-stroke groups. Second, we evaluate action recognition on the NTU RGB+D dataset. Across both settings, ES-VAE consistently outperforms standard VAEs and a range of sequence modeling baselines, including temporal convolutional networks, transformers, and graph convolutional networks. More broadly, ES-VAE provides a principled framework for learning generative models of longitudinal data on pose shape manifolds, offering improved latent representation and downstream performance compared to existing deep learning approaches.

Related papers

• KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generation [28.822034731724013]
  Articulated objects, such as laptops and drawers, exhibit significant challenges for 3D reconstruction and pose estimation.<n>We propose KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generation.
  arXiv  Detail & Related papers  (2025-10-20T04:15:40Z)
• MORPH: Shape-agnostic PDE Foundation Models [37.26306668589026]
  MORPH is a shape-agnostic, autoregressive foundation model for partial differential equations (PDEs)<n>It is built on a convolutional vision backbone that seamlessly handles heterogeneous evaluations of varying data dimensionality (1D--3D)<n>Across extensive datasets, MORPH matches or surpasses strong baselines and recent state-of-the-art models.
  arXiv  Detail & Related papers  (2025-09-25T22:38:36Z)
• Geometric Operator Learning with Optimal Transport [77.16909146519227]
  We propose integrating optimal transport (OT) into operator learning for partial differential equations (PDEs) on complex geometries.<n>For 3D simulations focused on surfaces, our OT-based neural operator embeds the surface geometry into a 2D parameterized latent space.<n> Experiments with Reynolds-averaged Navier-Stokes equations (RANS) on the ShapeNet-Car and DrivAerNet-Car datasets show that our method achieves better accuracy and also reduces computational expenses.
  arXiv  Detail & Related papers  (2025-07-26T21:28:25Z)
• Deformable registration and generative modelling of aortic anatomies by auto-decoders and neural ODEs [4.299467172990462]
  This work introduces AD-SVFD, a deep learning model for the registration of shapes to a pre-defined reference and for the generation of synthetic anatomies.<n>A distinctive feature of AD-SVFD is its auto-decoder structure, that enables generalization across shape cohorts and favors efficient weight sharing.<n>The use of implicit shape representations enables generative applications: new anatomies can be synthesized by suitably sampling from the latent space and applying the corresponding inverse transformations to the reference geometry.
  arXiv  Detail & Related papers  (2025-06-01T10:30:58Z)
• Single-shot prediction of parametric partial differential equations [3.987215131970378]
  Flexi-VAE is a data-driven framework for efficient single-shot forecasting of parametric partial differential equations (PDEs)<n>We introduce a neural propagator that advances latent viscous forward in time, aligning latent evolution with physical state reconstruction in a variational autoencoder setting.<n>We validate Flexi-VAE on PDE benchmarks, the 1D Burgers equation and the 2D advection-diffusion equation, achieving accurate forecasts across wide parametric ranges.
  arXiv  Detail & Related papers  (2025-05-14T01:48:26Z)
• A Geometry-Aware Message Passing Neural Network for Modeling Aerodynamics over Airfoils [61.60175086194333]
  aerodynamics is a key problem in aerospace engineering, often involving flows interacting with solid objects such as airfoils.<n>Here, we consider modeling of incompressible flows over solid objects, wherein geometric structures are a key factor in determining aerodynamics.<n>To effectively incorporate geometries, we propose a message passing scheme that efficiently and expressively integrates the airfoil shape with the mesh representation.<n>These design choices lead to a purely data-driven machine learning framework known as GeoMPNN, which won the Best Student Submission award at the NeurIPS 2024 ML4CFD Competition, placing 4th overall.
  arXiv  Detail & Related papers  (2024-12-12T16:05:39Z)
• Automatic Parameterization for Aerodynamic Shape Optimization via Deep Geometric Learning [60.69217130006758]
  We propose two deep learning models that fully automate shape parameterization for aerodynamic shape optimization. Both models are optimized to parameterize via deep geometric learning to embed human prior knowledge into learned geometric patterns. We perform shape optimization experiments on 2D airfoils and discuss the applicable scenarios for the two models.
  arXiv  Detail & Related papers  (2023-05-03T13:45:40Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion [15.359134407309726]
  Our model decomposes highly correlated skeleton data into a set of few spatial basis of switching temporal processes. This results in a dynamical deep generative latent model that parses the meaningful intrinsic states in the dynamics of 3D pose data.
  arXiv  Detail & Related papers  (2021-06-18T23:58:49Z)
• Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model [58.17021225930069]
  We explain the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) We propose a more efficient EAT model, and design task-related heads to deal with different tasks more flexibly. Our approach achieves state-of-the-art results on the ImageNet classification task compared with recent vision transformer works.
  arXiv  Detail & Related papers  (2021-05-31T16:20:03Z)
• Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency [114.02182755620784]
  We present an end-to-end joint training framework that explicitly models 6-DoF motion of multiple dynamic objects, ego-motion and depth in a monocular camera setup without supervision. Our framework is shown to outperform the state-of-the-art depth and motion estimation methods.
  arXiv  Detail & Related papers  (2021-02-04T14:26:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.