Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors

• URL: http://arxiv.org/abs/2312.02196v2
• Date: Thu, 7 Mar 2024 07:07:20 GMT
• Title: Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors
• Authors: Yu Zhang, Songpengcheng Xia, Lei Chu, Jiarui Yang, Qi Wu, Ling Pei
• Abstract summary: This paper introduces a novel human pose estimation approach using sparse inertial sensors. It leverages a diverse array of real inertial motion capture data from different skeleton formats to improve motion diversity and model generalization. The approach demonstrates superior performance over state-of-the-art models across five public datasets, notably reducing pose error by 19% on the DIP-IMU dataset.
• Score: 17.3834029178939
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This paper introduces a novel human pose estimation approach using sparse inertial sensors, addressing the shortcomings of previous methods reliant on synthetic data. It leverages a diverse array of real inertial motion capture data from different skeleton formats to improve motion diversity and model generalization. This method features two innovative components: a pseudo-velocity regression model for dynamic motion capture with inertial sensors, and a part-based model dividing the body and sensor data into three regions, each focusing on their unique characteristics. The approach demonstrates superior performance over state-of-the-art models across five public datasets, notably reducing pose error by 19\% on the DIP-IMU dataset, thus representing a significant improvement in inertial sensor-based human pose estimation. Our codes are available at {\url{https://github.com/dx118/dynaip}}.

Related papers

• MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds [20.83684434910106]
  We present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Specifically, we propose novel decoupled joint acceleration to model human dynamics from existing limited motion data. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks.
  arXiv  Detail & Related papers  (2024-09-01T15:00:16Z)
• Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation [5.143097874851516]
  We study two novel methods tailored for multi-sensor scenarios, namely Input Sensor Dropout (ISensD) and Ensemble Sensor Invariant (ESensI) We demonstrate that these methods effectively increase the robustness of model predictions to missing sensors. We observe that ensemble multi-sensor models are the most robust to the lack of sensors.
  arXiv  Detail & Related papers  (2024-07-22T09:58:29Z)
• Sensor Data Augmentation from Skeleton Pose Sequences for Improving Human Activity Recognition [5.669438716143601]
  Human Activity Recognition (HAR) has not fully capitalized on the proliferation of deep learning. We propose a novel approach to improve wearable sensor-based HAR by introducing a pose-to-sensor network model. Our contributions include the integration of simultaneous training, direct pose-to-sensor generation, and a comprehensive evaluation on the MM-Fit dataset.
  arXiv  Detail & Related papers  (2024-04-25T10:13:18Z)
• SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers [57.46911575980854]
  We introduce SkelFormer, a novel markerless motion capture pipeline for multi-view human pose and shape estimation. Our method first uses off-the-shelf 2D keypoint estimators, pre-trained on large-scale in-the-wild data, to obtain 3D joint positions. Next, we design a regression-based inverse-kinematic skeletal transformer that maps the joint positions to pose and shape representations from heavily noisy observations.
  arXiv  Detail & Related papers  (2024-04-19T04:51:18Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• Full-Body Motion Reconstruction with Sparse Sensing from Graph Perspective [22.761692765158646]
  Estimating 3D full-body pose from sparse sensor data is a pivotal technique employed for the reconstruction of realistic human motions in Augmented Reality and Virtual Reality. We use Body Pose Graph (BPG) to represent the human body and translate the challenge into a prediction problem of graph missing nodes. Our method's effectiveness is evidenced by the attained state-of-the-art performance, particularly in lower body motion, outperforming other baseline methods.
  arXiv  Detail & Related papers  (2024-01-22T09:29:42Z)
• EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision [85.17951804790515]
  EmerNeRF is a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes. It simultaneously captures scene geometry, appearance, motion, and semantics via self-bootstrapping. Our method achieves state-of-the-art performance in sensor simulation.
  arXiv  Detail & Related papers  (2023-11-03T17:59:55Z)
• Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs [79.72586714047199]
  We propose an attention-based deep learning method to reconstruct full-body motion from six IMU sensors in real-time. Our method achieves new state-of-the-art results both quantitatively and qualitatively, while being simple to implement and smaller in size.
  arXiv  Detail & Related papers  (2022-03-29T16:24:52Z)
• H4D: Human 4D Modeling by Learning Neural Compositional Representation [75.34798886466311]
  This work presents a novel framework that can effectively learn a compact and compositional representation for dynamic human. A simple yet effective linear motion model is proposed to provide a rough and regularized motion estimation. Experiments demonstrate our method is not only efficacy in recovering dynamic human with accurate motion and detailed geometry, but also amenable to various 4D human related tasks.
  arXiv  Detail & Related papers  (2022-03-02T17:10:49Z)
• Dynamical Deep Generative Latent Modeling of 3D Skeletal Motion [15.359134407309726]
  Our model decomposes highly correlated skeleton data into a set of few spatial basis of switching temporal processes. This results in a dynamical deep generative latent model that parses the meaningful intrinsic states in the dynamics of 3D pose data.
  arXiv  Detail & Related papers  (2021-06-18T23:58:49Z)
• HuMoR: 3D Human Motion Model for Robust Pose Estimation [100.55369985297797]
  HuMoR is a 3D Human Motion Model for Robust Estimation of temporal pose and shape. We introduce a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. We demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset.
  arXiv  Detail & Related papers  (2021-05-10T21:04:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.