Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation

• URL: http://arxiv.org/abs/2607.18875v1
• Date: Tue, 21 Jul 2026 09:06:31 GMT
• Title: Wave2Body: Rethinking mmWave Human Pose Estimation as Radar-to-Body Token Translation
• Abstract summary: Millimeter-wave (mmWave) radar enables privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation.<n>Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their alignment.<n>We propose Wave2Body, a radar-to-body token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them.
• Score: 17.79621154844076
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Millimeter-wave (mmWave) radar enables privacy-friendly human sensing, but its sparse point clouds are physical measurements of view-dependent electromagnetic reflections and only indirectly characterize body articulation. Recovering a complete 3D pose from such partial, geometry-dependent observations is therefore under-constrained. Existing methods directly regress joint coordinates from paired radar-pose data, relying on the same limited paired supervision to learn radar perception, human-body structure, and their alignment. This coupling can encourage dataset-specific shortcuts under ambiguous radar observations. We propose Wave2Body, a radar-to-body token translation framework that decouples these learning targets using a self-supervised mmWave tokenizer, a pretrained compositional body tokenizer that defines the output space, and a lightweight translator between them. Experiments on M4Human and mmBody show that Wave2Body achieves stronger cross-domain generalization than previous methods while incurring much lower computational costs for training and inference. All the code and experiment results are publicly available at https://github.com/Galaxywalk/Wave2Body.

Related papers

• ROMNet: a hybrid reduced order modeling and machine learning approach to waveform inversion [53.79823394212125]
  Waveform inversion seeks to estimate the wave speed of a heterogeneous, inaccessible medium.<n>We introduce the methodology, called ROMNet, and test it with numerical simulations.<n>We compare the performance of ROMNet with the direct ROM-based inversion and with two representative deep learning approaches to FWI.
  arXiv  Detail & Related papers  (2026-08-25T21:13:39Z)
• Teaching Foundation Models to Read mmWave: Pose-Guided Kinematic Representation for Human Behavior Understanding [23.73222950886392]
  Millimeter-wave (mmWave) radar provides a privacy-friendly and contactless sensing modality.<n>Existing radar-language methods often rely on synthetic data or lack explicit supervision for human structure and motion dynamics.<n>We present mmMind, a radar-language model that uses a synchronized 3 pose training model.
  arXiv  Detail & Related papers  (2026-08-04T18:25:55Z)
• Dense Soft Weighting for Radar Ego-Velocity Estimation [6.356174487225252]
  We present Dense Soft Weighting, an analytic radar front-end for ego-velocity estimation.<n>It maps every range-Doppler cell to a continuous confidence metric rather than enforcing a binary detection threshold.<n>It reduces mean absolute pose error by 31-45% relative to the strongest CFAR point-cloud baseline.
  arXiv  Detail & Related papers  (2026-07-29T14:37:52Z)
• A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery [13.982684122303896]
  We propose a two-stage framework for radar-based human body reconstruction.<n>First, we introduce a human reflection extraction module that performs coarse-to-fine localization.<n>Second, we design a motion-aware mesh recovery network that reconstructs the human body.
  arXiv  Detail & Related papers  (2026-05-08T22:34:03Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation [14.466573808593887]
  RaCalNet is a novel framework that eliminates the need for dense supervision by using sparse LiDAR to supervise the learning of refined radar measurements.<n>RaCalNet produces depth maps with clear object contours and fine-grained textures, demonstrating superior visual quality compared to state-of-the-art dense-supervised methods.
  arXiv  Detail & Related papers  (2025-06-18T15:35:16Z)
• RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence [10.115852646162843]
  We present Radar-LLM, the first framework that leverages large language models (LLMs) for human understanding using millimeter-wave radar as the sensing modality.<n>To address data scarcity, we introduce a physics-aware pipeline synthesis that generates realistic radar-text pairs from motion-text datasets.<n>Radar-LLM achieves state-of-the-art performance across both synthetic and real-world benchmarks, enabling accurate translation of millimeter-wave signals to natural language descriptions.
  arXiv  Detail & Related papers  (2025-04-14T04:18:25Z)
• Skeleton Detection Using Dual Radars with Integration of Dual-View CNN Models and mmPose [0.0]
  This research proposes three Dual ViewCNN models, combining PointNet and mmPose, employing two mmWave radars.<n>While the proposed model shows suboptimal results for random walking, it excels in the arm swing case.
  arXiv  Detail & Related papers  (2024-11-28T16:40:58Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• Bi-LRFusion: Bi-Directional LiDAR-Radar Fusion for 3D Dynamic Object Detection [78.59426158981108]
  We introduce a bi-directional LiDAR-Radar fusion framework, termed Bi-LRFusion, to tackle the challenges and improve 3D detection for dynamic objects. We conduct extensive experiments on nuScenes and ORR datasets, and show that our Bi-LRFusion achieves state-of-the-art performance for detecting dynamic objects.
  arXiv  Detail & Related papers  (2023-06-02T10:57:41Z)
• Semantic Segmentation of Radar Detections using Convolutions on Point Clouds [59.45414406974091]
  We introduce a deep-learning based method to convolve radar detections into point clouds. We adapt this algorithm to radar-specific properties through distance-dependent clustering and pre-processing of input point clouds. Our network outperforms state-of-the-art approaches that are based on PointNet++ on the task of semantic segmentation of radar point clouds.
  arXiv  Detail & Related papers  (2023-05-22T07:09:35Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.