milliMamba: Specular-Aware Human Pose Estimation via Dual mmWave Radar with Multi-Frame Mamba Fusion

• URL: http://arxiv.org/abs/2512.20128v1
• Date: Tue, 23 Dec 2025 07:40:25 GMT
• Title: milliMamba: Specular-Aware Human Pose Estimation via Dual mmWave Radar with Multi-Frame Mamba Fusion
• Authors: Niraj Prakash Kini, Shiau-Rung Tsai, Guan-Hsun Lin, Wen-Hsiao Peng, Ching-Wen Ma, Jenq-Neng Hwang,
• Abstract summary: We present a radar-based 2D human pose estimation framework.<n>We use a Cross-View Fusion Mamba to efficiently extract features from longer sequences.<n>We also incorporate a velocity loss alongside standard keypoint loss during training.
• Score: 24.89937570181235
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Millimeter-wave radar offers a privacy-preserving and lighting-invariant alternative to RGB sensors for Human Pose Estimation (HPE) task. However, the radar signals are often sparse due to specular reflection, making the extraction of robust features from radar signals highly challenging. To address this, we present milliMamba, a radar-based 2D human pose estimation framework that jointly models spatio-temporal dependencies across both the feature extraction and decoding stages. Specifically, given the high dimensionality of radar inputs, we adopt a Cross-View Fusion Mamba encoder to efficiently extract spatio-temporal features from longer sequences with linear complexity. A Spatio-Temporal-Cross Attention decoder then predicts joint coordinates across multiple frames. Together, this spatio-temporal modeling pipeline enables the model to leverage contextual cues from neighboring frames and joints to infer missing joints caused by specular reflections. To reinforce motion smoothness, we incorporate a velocity loss alongside the standard keypoint loss during training. Experiments on the TransHuPR and HuPR datasets demonstrate that our method achieves significant performance improvements, exceeding the baselines by 11.0 AP and 14.6 AP, respectively, while maintaining reasonable complexity. Code: https://github.com/NYCU-MAPL/milliMamba

Related papers

• SkeFi: Cross-Modal Knowledge Transfer for Wireless Skeleton-Based Action Recognition [20.020503149009787]
  Existing solutions use RGB cameras to annotate skeletal keypoints, but their performance declines in dark environments and raises privacy concerns.<n>This paper explores non-invasive wireless sensors, i.e., LiDAR and mmWave, to mitigate these challenges.<n> Experiments demonstrate that SkeFi realizes state-of-the-art performances on mmWave and LiDAR.
  arXiv  Detail & Related papers  (2026-01-18T14:39:02Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• TacoDepth: Towards Efficient Radar-Camera Depth Estimation with One-stage Fusion [54.46664104437454]
  We propose TacoDepth, an efficient and accurate Radar-Camera depth estimation model with one-stage fusion.<n>Specifically, the graph-based Radar structure extractor and the pyramid-based Radar fusion module are designed.<n>Compared with the previous state-of-the-art approach, TacoDepth improves depth accuracy and processing speed by 12.8% and 91.8%.
  arXiv  Detail & Related papers  (2025-04-16T05:25:04Z)
• RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence [10.115852646162843]
  We present Radar-LLM, the first framework that leverages large language models (LLMs) for human understanding using millimeter-wave radar as the sensing modality.<n>To address data scarcity, we introduce a physics-aware pipeline synthesis that generates realistic radar-text pairs from motion-text datasets.<n>Radar-LLM achieves state-of-the-art performance across both synthetic and real-world benchmarks, enabling accurate translation of millimeter-wave signals to natural language descriptions.
  arXiv  Detail & Related papers  (2025-04-14T04:18:25Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection [0.5898893619901381]
  This paper proposes MambaST, a plug-and-play cross-spectral spatial-temporal fusion pipeline for efficient pedestrian detection. It is difficult to perform accurate detection using RGB cameras under dark or low-light conditions. Our proposed model also achieves superior performance on small-scale pedestrian detection.
  arXiv  Detail & Related papers  (2024-08-02T06:20:48Z)
• G3R: Generating Rich and Fine-grained mmWave Radar Data from 2D Videos for Generalized Gesture Recognition [19.95047010486547]
  We develop a software pipeline that exploits wealthy 2D videos to generate realistic radar data. It addresses the challenge of simulating diversified and fine-grained reflection properties of user gestures. We implement and evaluate G3R using 2D videos from public data sources and self-collected real-world radar data.
  arXiv  Detail & Related papers  (2024-04-23T11:22:59Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds [59.45414406974091]
  We introduce a deep-learning based method to convolve radar detections into point clouds. We adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds.
  arXiv  Detail & Related papers  (2023-05-22T07:09:35Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.