MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video

• URL: http://arxiv.org/abs/2605.00242v1
• Date: Thu, 30 Apr 2026 21:23:03 GMT
• Title: MAEPose: Self-Supervised Spatiotemporal Learning for Human Pose Estimation on mmWave Video
• Abstract summary: We present MAEPose, a masked auto-encoding-based human pose estimation approach that operates directly on mmWave videos.<n>We evaluate MAEPose across three datasets based on leave-one-person-cross-out-validation with rigorous statistical testing.
• Score: 12.636232510548062
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Millimetre-wave (mmWave) radar offers a more privacy-preserving alternative to RGB-based human pose estimation. However, existing methods typically rely on pre-extracted intermediate representations such as sparse point clouds or spectrogram images, where the rich spatiotemporal information naturally present in radar video streams is discarded for model learning, while such signal processing adds system complexity. In addition, existing solutions are mainly conducted in an end-to-end supervised manner without leveraging unlabelled raw video streams to learn generalized representations. In this study, we present MAEPose, a masked autoencoding-based human pose estimation approach that operates directly on mmWave spectrogram videos. MAEPose learns spatiotemporal motion-aware generalized representations from unlabelled radar video, and leverages its heatmap decoder for multi-frame pose estimation predictions. We evaluate it across three datasets based on leave-one-person-out cross-validation with rigorous statistical testing. MAEPose consistently outperforms state-of-the-art baselines by up to 22.1% in MPJPE p<0.05, and maintains robust accuracy under zero-shot bystander interference with only a 6.5% error increase. Ablation studies confirm that both the pre-training and the heatmap decoder contribute substantially, while modality analysis indicates that leveraging Range-Doppler video as input achieves better pose estimation performance than Range-Azimuth or their fusion, with lower computational cost.

Related papers

• Dense Soft Weighting for Radar Ego-Velocity Estimation [6.356174487225252]
  We present Dense Soft Weighting, an analytic radar front-end for ego-velocity estimation.<n>It maps every range-Doppler cell to a continuous confidence metric rather than enforcing a binary detection threshold.<n>It reduces mean absolute pose error by 31-45% relative to the strongest CFAR point-cloud baseline.
  arXiv  Detail & Related papers  (2026-07-29T14:37:52Z)
• mmGAT: Pose Estimation by Graph Attention with Mutual Features from mmWave Radar Point Cloud [3.300544441511928]
  This paper exploits the capabilities of millimeter-wave (mmWave) radar technology for human pose estimation.<n>We present a unique feature extraction technique that exploits the full potential of the GNN processing method for pose estimation.<n>Our approach achieves a noteworthy reduction of pose estimation mean per joint position error (MPJPE) by 35.6% and PA-MPJPE by 14.1% from the current state of the art benchmark within this domain.
  arXiv  Detail & Related papers  (2026-03-09T16:15:45Z)
• Quality-Aware Framework for Video-Derived Respiratory Signals [10.760284839837395]
  Video-based respiratory rate estimation is often unreliable due to inconsistent signal quality across extraction methods.<n>We present a predictive, quality-aware framework that integrates heterogeneous signal sources with dynamic assessment of reliability.
  arXiv  Detail & Related papers  (2025-12-16T05:04:24Z)
• iGaussian: Real-Time Camera Pose Estimation via Feed-Forward 3D Gaussian Splatting Inversion [62.09575122593993]
  iGaussian is a two-stage feed-forward framework that achieves real-time camera pose estimation through direct 3D Gaussian inversion.<n> Experimental results on the NeRF Synthetic, Mip-NeRF 360, and T&T+DB datasets demonstrate a significant performance improvement over previous methods.
  arXiv  Detail & Related papers  (2025-11-18T05:22:22Z)
• A Foundation Model for DAS Signal Recognition and Visual Prompt Tuning of the Pre-trained Model for Downstream Tasks [6.14430079610632]
  This study proposes a foundational model for DAS signal recognition based on a Masked Autocoder, named MAEPD.<n>The model is pretrained on a dataset of 635860 samples, encompassing DAS gait signals, 2temporal GASF images for perimeter security, 2D time-frequency images for pipeline leakage, and open-dataset signals including whale vocalizations and seismic activities.<n>The VPT-Deep approach achieves a classification accuracy of 96.94% with just 0.322% of parameters fine-tuned, surpassing the traditional Full Fine Tuning (FFT) method by 0.61% and reducing training time by
  arXiv  Detail & Related papers  (2025-08-06T11:02:25Z)
• ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion [10.269536505085524]
  This paper introduces a probability map guided multi-format feature fusion model, ProbRadarM3F.<n>ProbRadarM3F fuses the traditional heatmap features and the positional features, then effectively achieves the estimation of 14 keypoints of the human body.
  arXiv  Detail & Related papers  (2024-05-08T15:54:57Z)
• Coordinate Transformer: Achieving Single-stage Multi-person Mesh Recovery from Videos [91.44553585470688]
  Multi-person 3D mesh recovery from videos is a critical first step towards automatic perception of group behavior in virtual reality, physical therapy and beyond. We propose the Coordinate transFormer (CoordFormer) that directly models multi-person spatial-temporal relations and simultaneously performs multi-mesh recovery in an end-to-end manner. Experiments on the 3DPW dataset demonstrate that CoordFormer significantly improves the state-of-the-art, outperforming the previously best results by 4.2%, 8.8% and 4.7% according to the MPJPE, PAMPJPE, and PVE metrics, respectively.
  arXiv  Detail & Related papers  (2023-08-20T18:23:07Z)
• MAPS: A Noise-Robust Progressive Learning Approach for Source-Free Domain Adaptive Keypoint Detection [76.97324120775475]
  Cross-domain keypoint detection methods always require accessing the source data during adaptation. This paper considers source-free domain adaptive keypoint detection, where only the well-trained source model is provided to the target domain.
  arXiv  Detail & Related papers  (2023-02-09T12:06:08Z)
• TempNet: Temporal Attention Towards the Detection of Animal Behaviour in Videos [63.85815474157357]
  We propose an efficient computer vision- and deep learning-based method for the detection of biological behaviours in videos. TempNet uses an encoder bridge and residual blocks to maintain model performance with a two-staged, spatial, then temporal, encoder. We demonstrate its application to the detection of sablefish (Anoplopoma fimbria) startle events.
  arXiv  Detail & Related papers  (2022-11-17T23:55:12Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• Self-Attentive 3D Human Pose and Shape Estimation from Videos [82.63503361008607]
  We present a video-based learning algorithm for 3D human pose and shape estimation. We exploit temporal information in videos and propose a self-attention module. We evaluate our method on the 3DPW, MPI-INF-3DHP, and Human3.6M datasets.
  arXiv  Detail & Related papers  (2021-03-26T00:02:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.