Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking

• URL: http://arxiv.org/abs/2609.08346v1
• Date: Tue, 08 Sep 2026 07:17:33 GMT
• Title: Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking
• Abstract summary: We introduce RGBTR-Motion, a benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities.<n>We also develop SAM-Radar, an RGB, thermal, and radar-based segmentation and tracking framework built on SAM 3.
• Score: 18.63898447592778
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motion from appearance, optical flow, or estimated trajectories lose that evidence under poor illumination, adverse weather, reflections, and occlusion. Radar is a natural remedy because it measures radial velocity directly instead of inferring it from photometric correspondence. However, existing benchmarks do not jointly provide radar measurements, dense moving-instance masks, and temporally consistent identities for surveillance. We therefore introduce RGBTR-Motion, a synchronized and calibrated fixed-camera benchmark that pairs RGB, thermal, and radar streams with dense instance masks and temporally consistent identities across diverse surveillance scenes. We also develop SAM-Radar, an RGB, thermal, and radar-based segmentation and tracking framework built on SAM 3. SAM-Radar's radar-aware detector fuses calibrated RGBT features with radar returns that are grounded at their projected image locations, and motion supervision, implemented as foreground classification of those projected returns, teaches the detector to reject clutter without any text prompt. The tracker associates accepted radar returns with individual trajectories and uses them as physical evidence that a visually degraded target remains present. This allows it to bridge short periods of low visibility or occlusion and reconnect a reappearing target to its existing identity instead of starting a new track. SAM-Radar attains 0.7027 IoU and 0.8090 F1-50, and raises MOTA, HOTA, and IDF1 by 0.2977, 0.1603, and 0.2857 over the strongest competing values.

Related papers

• DiffRadar: Differentiable Physics-Aware Radar SLAM with Gaussian Fields [8.442884076695789]
  We present DiffRadar, a real-time radar SLAM system that models radar observations as a differentiable, physics-aware Gaussian field.<n>DiffRadar represents the scene as anisotropic Gaussian primitives and renders radar measurements in range-azimuth and Doppler-azimuth spaces.<n>We implement DiffRadar on commodity FMCW radar hardware and evaluate it on both the public Radarize benchmark and a controlled stress-test suite.
  arXiv  Detail & Related papers  (2026-07-14T02:08:14Z)
• RadarTwin: Scene-Specific mmWave Radar Simulation and Learning for Mobile Indoor Perception [2.0255467447384015]
  RadarTwin is a framework for generating deployment-specific radar training data before real data collection.<n>We show that simulated and real radar share the same object-discriminative shape and material features.<n>A representation trained on simulation alone recognizes real objects at 2.5 times chance with no real radar labels, and a few labeled examples raise this to 95.3% on a 12-way recognition task.
  arXiv  Detail & Related papers  (2026-06-24T05:50:26Z)
• RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields [49.41438378775056]
  Radars are an ideal complement to cameras, with cameras offering fine angular resolution and radars providing metric depth and robustness under adverse weather.<n>RadarSim produces sharper geometry and Doppler range frames than radar-only reconstructions.
  arXiv  Detail & Related papers  (2026-05-25T21:03:40Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds [25.36192517603375]
  We address moving instance tracking in sparse radar point clouds to enhance scene interpretation.<n>We propose a learning-based radar tracker incorporating temporal offset predictions to enable direct center-based association.<n>Our approach shows an improved performance on the moving instance tracking benchmark of the RadarScenes dataset.
  arXiv  Detail & Related papers  (2025-07-04T09:57:28Z)
• NOVA: Navigation via Object-Centric Visual Autonomy for High-Speed Target Tracking in Unstructured GPS-Denied Environments [56.35569661650558]
  We introduce NOVA, a fully onboard, object-centric framework that enables robust target tracking and collision-aware navigation.<n>Rather than constructing a global map, NOVA formulates perception, estimation, and control entirely in the target's reference frame.<n>We validate NOVA across challenging real-world scenarios, including urban mazes, forest trails, and repeated transitions through buildings with intermittent GPS loss.
  arXiv  Detail & Related papers  (2025-06-23T14:28:30Z)
• RobuRCDet: Enhancing Robustness of Radar-Camera Fusion in Bird's Eye View for 3D Object Detection [68.99784784185019]
  Poor lighting or adverse weather conditions degrade camera performance.<n>Radar suffers from noise and positional ambiguity.<n>We propose RobuRCDet, a robust object detection model in BEV.
  arXiv  Detail & Related papers  (2025-02-18T17:17:38Z)
• RETR: Multi-View Radar Detection Transformer for Indoor Perception [2.6913398550088483]
  Existing radar perception pipelines fail to account for distinctive characteristics of the multi-view radar setting.<n>We propose Radar dEtection TRansformer (RETR), an extension of the popular DETR architecture, tailored for multi-view radar perception.<n>Our approach outperforms existing state-of-the-art methods by a margin of 15.38+ AP for object detection and 11.91+ IoU for instance segmentation.
  arXiv  Detail & Related papers  (2024-11-15T15:51:25Z)
• Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar [62.51065633674272]
  We introduce Radar Fields - a neural scene reconstruction method designed for active radar imagers. Our approach unites an explicit, physics-informed sensor model with an implicit neural geometry and reflectance model to directly synthesize raw radar measurements. We validate the effectiveness of the method across diverse outdoor scenarios, including urban scenes with dense vehicles and infrastructure.
  arXiv  Detail & Related papers  (2024-05-07T20:44:48Z)
• Radar Artifact Labeling Framework (RALF): Method for Plausible Radar Detections in Datasets [2.5899040911480187]
  We propose a cross sensor Radar Artifact Labeling Framework (RALF) for labeling sparse radar point clouds. RALF provides plausibility labels for radar raw detections, distinguishing between artifacts and targets. We validate the results by evaluating error metrics on semi-manually labeled ground truth dataset of $3.28cdot106$ points.
  arXiv  Detail & Related papers  (2020-12-03T15:11:31Z)
• RadarNet: Exploiting Radar for Robust Perception of Dynamic Objects [73.80316195652493]
  We tackle the problem of exploiting Radar for perception in the context of self-driving cars. We propose a new solution that exploits both LiDAR and Radar sensors for perception. Our approach, dubbed RadarNet, features a voxel-based early fusion and an attention-based late fusion.
  arXiv  Detail & Related papers  (2020-07-28T17:15:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.