LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment

• URL: http://arxiv.org/abs/2402.17171v1
• Date: Tue, 27 Feb 2024 03:08:44 GMT
• Title: LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment
• Authors: Yiming Ren, Xiao Han, Chengfeng Zhao, Jingya Wang, Lan Xu, Jingyi Yu, Yuexin Ma
• Abstract summary: We present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation. We propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses.
• Score: 59.320414108383055
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For human-centric large-scale scenes, fine-grained modeling for 3D human global pose and shape is significant for scene understanding and can benefit many real-world applications. In this paper, we present LiveHPS, a novel single-LiDAR-based approach for scene-level human pose and shape estimation without any limitation of light conditions and wearable devices. In particular, we design a distillation mechanism to mitigate the distribution-varying effect of LiDAR point clouds and exploit the temporal-spatial geometric and dynamic information existing in consecutive frames to solve the occlusion and noise disturbance. LiveHPS, with its efficient configuration and high-quality output, is well-suited for real-world applications. Moreover, we propose a huge human motion dataset, named FreeMotion, which is collected in various scenarios with diverse human poses, shapes and translations. It consists of multi-modal and multi-view acquisition data from calibrated and synchronized LiDARs, cameras, and IMUs. Extensive experiments on our new dataset and other public datasets demonstrate the SOTA performance and robustness of our approach. We will release our code and dataset soon.

Related papers

• Multi-Resolution Generative Modeling of Human Motion from Limited Data [3.5229503563299915]
  We present a generative model that learns to synthesize human motion from limited training sequences. The model adeptly captures human motion patterns by integrating skeletal convolution layers and a multi-scale architecture.
  arXiv  Detail & Related papers  (2024-11-25T15:36:29Z)
• DeBaRA: Denoising-Based 3D Room Arrangement Generation [22.96293773013579]
  We introduce DeBaRA, a score-based model specifically tailored for precise, controllable and flexible arrangement generation in a bounded environment. We demonstrate that by focusing on spatial attributes of objects, a single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement.
  arXiv  Detail & Related papers  (2024-09-26T23:18:25Z)
• LiveHPS++: Robust and Coherent Motion Capture in Dynamic Free Environment [17.832694508927407]
  We introduce LiveHPS++, an innovative and effective solution based on a single LiDAR system. Benefiting from three meticulously designed modules, our method can learn dynamic and kinematic features from human movements. Our method has proven to significantly surpass existing state-of-the-art methods across various datasets.
  arXiv  Detail & Related papers  (2024-07-13T10:04:45Z)
• Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling [70.34875558830241]
  We present a way for learning a-temporal (4D) embedding, based on semantic semantic gears to allow for stratified modeling of dynamic regions of rendering the scene. At the same time, almost for free, our tracking approach enables free-viewpoint of interest - a functionality not yet achieved by existing NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-06T03:37:39Z)
• Scaling Up Dynamic Human-Scene Interaction Modeling [58.032368564071895]
  TRUMANS is the most comprehensive motion-captured HSI dataset currently available. It intricately captures whole-body human motions and part-level object dynamics. We devise a diffusion-based autoregressive model that efficiently generates HSI sequences of any length.
  arXiv  Detail & Related papers  (2024-03-13T15:45:04Z)
• WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting [38.95768804529958]
  We augment the Open Motion dataset with large-scale, high-quality, diverse LiDAR data for the motion forecasting task. The new augmented dataset WOMD-LiDAR consists of over 100,000 scenes that each spans 20 seconds, consisting of well-synchronized and high quality LiDAR point clouds captured across a range of urban and suburban geographies. Experiments show that the LiDAR data brings improvement in the motion forecasting task.
  arXiv  Detail & Related papers  (2023-04-07T20:23:15Z)
• Progressive Multi-view Human Mesh Recovery with Self-Supervision [68.60019434498703]
  Existing solutions typically suffer from poor generalization performance to new settings. We propose a novel simulation-based training pipeline for multi-view human mesh recovery.
  arXiv  Detail & Related papers  (2022-12-10T06:28:29Z)
• LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors [38.60837840737258]
  We propose a multi-sensor fusion method for capturing 3D human motions with accurate consecutive local poses and global trajectories in large-scale scenarios. We design a two-stage pose estimator in a coarse-to-fine manner, where point clouds provide the coarse body shape and IMU measurements optimize the local actions. We collect a LiDAR-IMU multi-modal mocap dataset, LIPD, with diverse human actions in long-range scenarios.
  arXiv  Detail & Related papers  (2022-05-30T20:15:11Z)
• Transformer Inertial Poser: Attention-based Real-time Human Motion Reconstruction from Sparse IMUs [79.72586714047199]
  We propose an attention-based deep learning method to reconstruct full-body motion from six IMU sensors in real-time. Our method achieves new state-of-the-art results both quantitatively and qualitatively, while being simple to implement and smaller in size.
  arXiv  Detail & Related papers  (2022-03-29T16:24:52Z)
• HSPACE: Synthetic Parametric Humans Animated in Complex Environments [67.8628917474705]
  We build a large-scale photo-realistic dataset, Human-SPACE, of animated humans placed in complex indoor and outdoor environments. We combine a hundred diverse individuals of varying ages, gender, proportions, and ethnicity, with hundreds of motions and scenes, in order to generate an initial dataset of over 1 million frames. Assets are generated automatically, at scale, and are compatible with existing real time rendering and game engines.
  arXiv  Detail & Related papers  (2021-12-23T22:27:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.