ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion

• URL: http://arxiv.org/abs/2405.05164v2
• Date: Fri, 28 Jun 2024 06:11:11 GMT
• Title: ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion
• Authors: Bing Zhu, Zixin He, Weiyi Xiong, Guanhua Ding, Jianan Liu, Tao Huang, Wei Chen, Wei Xiang,
• Abstract summary: This paper introduces a probability map guided multi-format feature fusion model, ProbRadarM3F. ProbRadarM3F fuses the traditional heatmap features and the positional features, then effectively achieves the estimation of 14 keypoints of the human body.
• Score: 14.83158440666821
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Millimeter wave (mmWave) radar is a non-intrusive privacy and relatively convenient and inexpensive device, which has been demonstrated to be applicable in place of RGB cameras in human indoor pose estimation tasks. However, mmWave radar relies on the collection of reflected signals from the target, and the radar signals containing information is difficult to be fully applied. This has been a long-standing hindrance to the improvement of pose estimation accuracy. To address this major challenge, this paper introduces a probability map guided multi-format feature fusion model, ProbRadarM3F. This is a novel radar feature extraction framework using a traditional FFT method in parallel with a probability map based positional encoding method. ProbRadarM3F fuses the traditional heatmap features and the positional features, then effectively achieves the estimation of 14 keypoints of the human body. Experimental evaluation on the HuPR dataset proves the effectiveness of the model proposed in this paper, outperforming other methods experimented on this dataset with an AP of 69.9 %. The emphasis of our study is focusing on the position information that is not exploited before in radar singal. This provides direction to investigate other potential non-redundant information from mmWave rader.

Related papers

• CaFNet: A Confidence-Driven Framework for Radar Camera Depth Estimation [6.9404362058736995]
  This paper introduces a two-stage, end-to-end trainable Confidence-aware Fusion Net (CaFNet) for dense depth estimation. The first stage addresses radar-specific challenges, such as ambiguous elevation and noisy measurements. For the final depth estimation, we innovate a confidence-aware gated fusion mechanism to integrate radar and image features effectively.
  arXiv  Detail & Related papers  (2024-06-30T13:39:29Z)
• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Depth Estimation fusing Image and Radar Measurements with Uncertain Directions [14.206589791912458]
  In prior radar-image fusion work, image features are merged with the uncertain sparse depths measured by radar through convolutional layers. Our method avoids this problem by computing features only with an image and conditioning the features pixelwise with the radar depth. Our method improves training data by learning only these possibly correct radar directions, while the previous method trains raw radar measurements.
  arXiv  Detail & Related papers  (2024-03-23T10:16:36Z)
• Multistatic-Radar RCS-Signature Recognition of Aerial Vehicles: A Bayesian Fusion Approach [10.908489565519211]
  Radar Automated Target Recognition (RATR) for Unmanned Aerial Vehicles (UAVs) involves transmitting Electromagnetic Waves (EMWs) and performing target type recognition on the received radar echo. Previous studies highlighted the advantages of multistatic radar configurations over monostatic ones in RATR. We propose a fully Bayesian RATR framework employing Optimal Bayesian Fusion (OBF) to aggregate classification probability vectors from multiple radars.
  arXiv  Detail & Related papers  (2024-02-28T02:11:47Z)
• Radio Map Estimation -- An Open Dataset with Directive Transmitter Antennas and Initial Experiments [49.61405888107356]
  We release a dataset of simulated path loss radio maps together with realistic city maps from real-world locations and aerial images from open datasources. Initial experiments regarding model architectures, input feature design and estimation of radio maps from aerial images are presented.
  arXiv  Detail & Related papers  (2024-01-12T14:56:45Z)
• Echoes Beyond Points: Unleashing the Power of Raw Radar Data in Multi-modality Fusion [74.84019379368807]
  We propose a novel method named EchoFusion to skip the existing radar signal processing pipeline. Specifically, we first generate the Bird's Eye View (BEV) queries and then take corresponding spectrum features from radar to fuse with other sensors.
  arXiv  Detail & Related papers  (2023-07-31T09:53:50Z)
• HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar [30.51398364813315]
  This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR) This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation.
  arXiv  Detail & Related papers  (2022-10-22T22:28:40Z)
• LiRaNet: End-to-End Trajectory Prediction using Spatio-Temporal Radar Fusion [52.59664614744447]
  We present LiRaNet, a novel end-to-end trajectory prediction method which utilizes radar sensor information along with widely used lidar and high definition (HD) maps. automotive radar provides rich, complementary information, allowing for longer range vehicle detection as well as instantaneous velocity measurements.
  arXiv  Detail & Related papers  (2020-10-02T00:13:00Z)
• Depth Estimation from Monocular Images and Sparse Radar Data [93.70524512061318]
  In this paper, we explore the possibility of achieving a more accurate depth estimation by fusing monocular images and Radar points using a deep neural network. We find that the noise existing in Radar measurements is one of the main key reasons that prevents one from applying the existing fusion methods. The experiments are conducted on the nuScenes dataset, which is one of the first datasets which features Camera, Radar, and LiDAR recordings in diverse scenes and weather conditions.
  arXiv  Detail & Related papers  (2020-09-30T19:01:33Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.