Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion

• URL: http://arxiv.org/abs/2106.10393v1
• Date: Fri, 18 Jun 2021 23:58:49 GMT
• Title: Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion
• Authors: Amirreza Farnoosh, Sarah Ostadabbas
• Abstract summary: Our model decomposes highly correlated skeleton data into a set of few spatial basis of switching temporal processes. This results in a dynamical deep generative latent model that parses the meaningful intrinsic states in the dynamics of 3D pose data.
• Score: 15.359134407309726
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In this paper, we propose a Bayesian switching dynamical model for segmentation of 3D pose data over time that uncovers interpretable patterns in the data and is generative. Our model decomposes highly correlated skeleton data into a set of few spatial basis of switching temporal processes in a low-dimensional latent framework. We parameterize these temporal processes with regard to a switching deep vector autoregressive prior in order to accommodate both multimodal and higher-order nonlinear inter-dependencies. This results in a dynamical deep generative latent model that parses the meaningful intrinsic states in the dynamics of 3D pose data using approximate variational inference, and enables a realistic low-level dynamical generation and segmentation of complex skeleton movements. Our experiments on four biological motion data containing bat flight, salsa dance, walking, and golf datasets substantiate superior performance of our model in comparison with the state-of-the-art methods.

Related papers

• Equivariant Graph Neural Operator for Modeling 3D Dynamics [148.98826858078556]
  We propose Equivariant Graph Neural Operator (EGNO) to directly models dynamics as trajectories instead of just next-step prediction. EGNO explicitly learns the temporal evolution of 3D dynamics where we formulate the dynamics as a function over time and learn neural operators to approximate it. Comprehensive experiments in multiple domains, including particle simulations, human motion capture, and molecular dynamics, demonstrate the significantly superior performance of EGNO against existing methods.
  arXiv  Detail & Related papers  (2024-01-19T21:50:32Z)
• EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision [85.17951804790515]
  EmerNeRF is a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes. It simultaneously captures scene geometry, appearance, motion, and semantics via self-bootstrapping. Our method achieves state-of-the-art performance in sensor simulation.
  arXiv  Detail & Related papers  (2023-11-03T17:59:55Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• Discovering Dynamic Patterns from Spatiotemporal Data with Time-Varying Low-Rank Autoregression [12.923271427789267]
  We develop a time-reduced-rank vector autoregression model whose coefficient are parameterized by low-rank tensor factorization. In the temporal context, the complex time-varying system behaviors can be revealed by the temporal modes in the proposed model.
  arXiv  Detail & Related papers  (2022-11-28T15:59:52Z)
• Data-driven low-dimensional dynamic model of Kolmogorov flow [0.0]
  Reduced order models (ROMs) that capture flow dynamics are of interest for decreasing computational costs for simulation. This work presents a data-driven framework for minimal-dimensional models that effectively capture the dynamics and properties of the flow. We apply this to Kolmogorov flow in a regime consisting of chaotic and intermittent behavior.
  arXiv  Detail & Related papers  (2022-10-29T23:05:39Z)
• Gait Recognition in the Wild with Multi-hop Temporal Switch [81.35245014397759]
  gait recognition in the wild is a more practical problem that has attracted the attention of the community of multimedia and computer vision. This paper presents a novel multi-hop temporal switch method to achieve effective temporal modeling of gait patterns in real-world scenes.
  arXiv  Detail & Related papers  (2022-09-01T10:46:09Z)
• Scene Synthesis via Uncertainty-Driven Attribute Synchronization [52.31834816911887]
  This paper introduces a novel neural scene synthesis approach that can capture diverse feature patterns of 3D scenes. Our method combines the strength of both neural network-based and conventional scene synthesis approaches.
  arXiv  Detail & Related papers  (2021-08-30T19:45:07Z)
• Analysis of ODE2VAE with Examples [0.0]
  Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE) is a deep latent variable model. We show that the model is able to learn meaningful latent representations to an extent without any supervision.
  arXiv  Detail & Related papers  (2021-08-10T20:12:26Z)
• Multi-frame sequence generator of 4D human body motion [0.0]
  We propose a generative auto-encoder-based framework, which encodes, global locomotion including translation and rotation, and multi-frame temporal motion as a single latent space vector. Our results validate the ability of the model to reconstruct 4D sequences of human morphology within a low error bound. We also illustrate the benefits of the approach for 4D human motion prediction of future frames from initial human frames.
  arXiv  Detail & Related papers  (2021-06-07T13:56:46Z)
• Graph-based Normalizing Flow for Human Motion Generation and Reconstruction [20.454140530081183]
  We propose a probabilistic generative model to synthesize and reconstruct long horizon motion sequences conditioned on past information and control signals. We evaluate the models on a mixture of motion capture datasets of human locomotion with foot-step and bone-length analysis.
  arXiv  Detail & Related papers  (2021-04-07T09:51:15Z)
• Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition [79.33539539956186]
  We propose a simple method to disentangle multi-scale graph convolutions and a unified spatial-temporal graph convolutional operator named G3D. By coupling these proposals, we develop a powerful feature extractor named MS-G3D based on which our model outperforms previous state-of-the-art methods on three large-scale datasets.
  arXiv  Detail & Related papers  (2020-03-31T11:28:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.