Continuous-Time Human Motion Field from Events

• URL: http://arxiv.org/abs/2412.01747v1
• Date: Mon, 02 Dec 2024 17:42:59 GMT
• Title: Continuous-Time Human Motion Field from Events
• Authors: Ziyun Wang, Ruijun Zhang, Zi-Yan Liu, Yufu Wang, Kostas Daniilidis,
• Abstract summary: This paper addresses the challenges of estimating a continuous-time human motion field from a stream of events. We use a recurrent feed-forward neural network to predict human motion in the latent space of possible human motions. We present the first work that replaces traditional discrete-time predictions with a continuous human motion field represented as a time-implicit function.
• Score: 26.355172872510238
• License:
• Abstract: This paper addresses the challenges of estimating a continuous-time human motion field from a stream of events. Existing Human Mesh Recovery (HMR) methods rely predominantly on frame-based approaches, which are prone to aliasing and inaccuracies due to limited temporal resolution and motion blur. In this work, we predict a continuous-time human motion field directly from events by leveraging a recurrent feed-forward neural network to predict human motion in the latent space of possible human motions. Prior state-of-the-art event-based methods rely on computationally intensive optimization across a fixed number of poses at high frame rates, which becomes prohibitively expensive as we increase the temporal resolution. In comparison, we present the first work that replaces traditional discrete-time predictions with a continuous human motion field represented as a time-implicit function, enabling parallel pose queries at arbitrary temporal resolutions. Despite the promises of event cameras, few benchmarks have tested the limit of high-speed human motion estimation. We introduce Beam-splitter Event Agile Human Motion Dataset-a hardware-synchronized high-speed human dataset to fill this gap. On this new data, our method improves joint errors by 23.8% compared to previous event human methods while reducing the computational time by 69%.

Related papers

• COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation [98.05046790227561]
  COIN is a control-inpainting motion diffusion prior that enables fine-grained control to disentangle human and camera motions. COIN outperforms the state-of-the-art methods in terms of global human motion estimation and camera motion estimation.
  arXiv  Detail & Related papers  (2024-08-29T10:36:29Z)
• Towards Practical Human Motion Prediction with LiDAR Point Clouds [15.715130864327792]
  We propose textitLiDAR-HMP, the first single-LiDAR-based 3D human motion prediction approach. LiDAR-HMP receives the raw LiDAR point cloud as input and forecasts future 3D human poses directly. Our method achieves state-of-the-art performance on two public benchmarks and demonstrates remarkable robustness and efficacy in real-world deployments.
  arXiv  Detail & Related papers  (2024-08-15T15:10:01Z)
• Aligning Human Motion Generation with Human Perceptions [51.831338643012444]
  We propose a data-driven approach to bridge the gap by introducing a large-scale human perceptual evaluation dataset, MotionPercept, and a human motion critic model, MotionCritic. Our critic model offers a more accurate metric for assessing motion quality and could be readily integrated into the motion generation pipeline.
  arXiv  Detail & Related papers  (2024-07-02T14:01:59Z)
• RecMoDiffuse: Recurrent Flow Diffusion for Human Motion Generation [5.535590461577558]
  RecMoDiffuse is a new recurrent diffusion formulation for temporal modelling. We demonstrate the effectiveness of RecMoDiffuse in the temporal modelling of human motion.
  arXiv  Detail & Related papers  (2024-06-11T11:25:37Z)
• AdvMT: Adversarial Motion Transformer for Long-term Human Motion Prediction [2.837740438355204]
  We present the Adversarial Motion Transformer (AdvMT), a novel model that integrates a transformer-based motion encoder and a temporal continuity discriminator. With adversarial training, our method effectively reduces the unwanted artifacts in predictions, thereby ensuring the learning of more realistic and fluid human motions.
  arXiv  Detail & Related papers  (2024-01-10T09:15:50Z)
• Decomposed Human Motion Prior for Video Pose Estimation via Adversarial Training [7.861513525154702]
  We propose to decompose holistic motion prior to joint motion prior, making it easier for neural networks to learn from prior knowledge. We also utilize a novel regularization loss to balance accuracy and smoothness introduced by motion prior. Our method achieves 9% lower PA-MPJPE and 29% lower acceleration error than previous methods tested on 3DPW.
  arXiv  Detail & Related papers  (2023-05-30T04:53:34Z)
• Towards Accurate Human Motion Prediction via Iterative Refinement [9.910719309846128]
  FreqMRN takes into account both the kinematic structure of the human body and the temporal smoothness nature of motion. We evaluate FreqMRN on several standard benchmark datasets, including Human3.6M, AMASS and 3DPW.
  arXiv  Detail & Related papers  (2023-05-08T03:43:51Z)
• Direct Dense Pose Estimation [138.56533828316833]
  Dense human pose estimation is the problem of learning dense correspondences between RGB images and the surfaces of human bodies. Prior dense pose estimation methods are all based on Mask R-CNN framework and operate in a top-down manner of first attempting to identify a bounding box for each person. We propose a novel alternative method for solving the dense pose estimation problem, called Direct Dense Pose (DDP)
  arXiv  Detail & Related papers  (2022-04-04T06:14:38Z)
• Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction [63.62263239934777]
  We conduct an indepth study on various pose representations with a focus on their effects on the motion prediction task. We propose a novel RNN architecture termed AHMR (Attentive Hierarchical Motion Recurrent network) for motion prediction. Our approach outperforms the state-of-the-art methods in short-term prediction and achieves much enhanced long-term prediction proficiency.
  arXiv  Detail & Related papers  (2021-12-30T10:45:22Z)
• Long Term Motion Prediction Using Keyposes [122.22758311506588]
  We argue that, to achieve long term forecasting, predicting human pose at every time instant is unnecessary. We call such poses "keyposes", and approximate complex motions by linearly interpolating between subsequent keyposes. We show that learning the sequence of such keyposes allows us to predict very long term motion, up to 5 seconds in the future.
  arXiv  Detail & Related papers  (2020-12-08T20:45:51Z)
• Perpetual Motion: Generating Unbounded Human Motion [61.40259979876424]
  We focus on long-term prediction; that is, generating long sequences of human motion that is plausible. We propose a model to generate non-deterministic, textitever-changing, perpetual human motion. We train this using a heavy-tailed function of the KL divergence of a white-noise Gaussian process, allowing latent sequence temporal dependency.
  arXiv  Detail & Related papers  (2020-07-27T21:50:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.