Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

• URL: http://arxiv.org/abs/2606.22195v1
• Date: Sat, 20 Jun 2026 19:07:22 GMT
• Title: Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning
• Abstract summary: We propose a vision-assisted OFDM-ISAC framework to overcome the data-association and resolution limits of single-modality ISAC.<n>The proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE on a Blender-rendered vehicular testbed.
• Score: 10.44632158114449
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Orthogonal frequency division multiplexing (OFDM)-based integrated sensing and communication (ISAC) systems commonly extract target parameters by peak-searching a delay-Doppler map (DDM) constructed from reflected pilots. In multi-target scenarios, this results in ambiguity: the DDM does not reveal which physical target produced which peak, and two targets within the same delay-Doppler resolution cell cannot be separated. We propose a vision-assisted OFDM-ISAC framework that resolves both limitations by fusing wireless and visual modalities. The transmitter encodes an onboard street-view image with deep joint source-channel coding (DeepJSCC) and transmits it over the same OFDM waveform used for sensing; the receiver reconstructs the image, runs a fine-tuned YOLOv5 detector and fuses the resulting per-target features (bounding-box coordinates and class labels) with the DDM and transmitter-receiver geometry through a learned multi-modal network. To stabilize training of the high dimensional delay and Doppler classifiers, we introduce a Kullback Leibler loss against triangular soft labels centered on the ground-truth bin. On a Blender-rendered vehicular testbed, the proposed framework achieves a 16 cm localization root mean square error (RMSE) and a 10.8 ns delay RMSE. An ablation study confirms that removing the visual modality causes a 60x degradation in localization. These results highlight the potential of vision to overcome the data-association and resolution limits of single-modality ISAC.

Related papers

• S$^3$AM: A Single-Stream SAM with Reliability-Calibrated Frequency Adapter for Multi-modal Salient Object Detection [74.15663758681849]
  Single-stream framework integrates reliability-calibrated frequency adaptation into the adopted SAM backbone for MSOD.<n>Experiments on RGB-D, RGB-T, and RGB-NIR salient object detection benchmarks validate that the proposed framework achieves competitive performance.
  arXiv  Detail & Related papers  (2026-08-18T08:05:00Z)
• Multi-scale Decomposed Convolution Refinement Network for Visible-Infrared Person Re-Identification [11.621464807854816]
  Visible-infrared person re-identification (VI-ReID) suffers from cross-modal discrepancies and limited discriminative capabilities.<n>We propose MDCRNet, a Multi-scale Decomposed Convolution Refinement Network that enhances cross-modal feature learning and discriminative metric learning.
  arXiv  Detail & Related papers  (2026-08-17T02:17:02Z)
• LLM-Empowered Multimodal Fusion Framework for Autonomous Driving: Semantic Enhancement and Channel-Adaptive Design [56.76240343283953]
  We propose a Large Language Model-centric Semantic-layer Channel-aware Integrated Perception framework.<n>It fuses a local visual stream with a quality-varying external radar stream used to cover perception-blind spots.<n>Experiments on nuScenes and VIRAT validate our approach.
  arXiv  Detail & Related papers  (2026-07-02T06:44:02Z)
• Echo-DM: Ultrasound Marker Removal via Conditional Latent Diffusion and Region-Aware Fusion [50.02433280054874]
  markers can introduce shortcut bias in downstream automated analysis.<n>We present Echo-DM, a framework for ultrasound marker removal via conditional latent diffusion and region-aware fusion.<n>Experiments on Echo-PAIR, a large-scale paired clinical ultrasound dataset, demonstrate superior marker removal and strong anatomical fidelity.
  arXiv  Detail & Related papers  (2026-06-08T11:56:33Z)
• DAP: Doppler-aware Point Network for Heterogeneous mmWave Action Recognition [42.97701141999221]
  Millimeter-wave (mmWave) radar provides privacy-preserving sensing and is valuable for human action recognition (HAR)<n>Existing mmWave point cloud datasets are limited in scale and mostly collected under single-source settings.<n>To address this, we introduce Millimeter-HAR, the largest and first mmWave point cloud HAR dataset for heterogeneous multi-source scenarios.
  arXiv  Detail & Related papers  (2026-05-10T15:34:53Z)
• SPIRIT: Adapting Vision Foundation Models for Unified Single- and Multi-Frame Infrared Small Target Detection [18.86422994684341]
  Infrared small target detection (IRSTD) is crucial for surveillance and early-warning, with deployments spanning both single-frame analysis and video-mode tracking.<n>We propose SPIRIT, a unified and VFM-compatible framework that adapts VFMs to IRSTD via lightweight physics-informed plug-ins.
  arXiv  Detail & Related papers  (2026-02-02T09:15:29Z)
• UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotations [3.588053519843616]
  We propose a parameter-efficient framework that adapts a single ImageNet-pretrained diffusion model to multiple sensing modalities using only target-domain data.<n>UniDiff combines FiLM-based timestep-modality conditioning, parameter-efficient adaptation of approximately 5% of parameters, and pseudo-RGB anchoring to preserve pre-trained representations and prevent catastrophic forgetting.
  arXiv  Detail & Related papers  (2025-11-29T00:57:53Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• Unsupervised Modality Adaptation with Text-to-Image Diffusion Models for Semantic Segmentation [54.96563068182733]
  We propose Modality Adaptation with text-to-image Diffusion Models (MADM) for semantic segmentation task. MADM utilizes text-to-image diffusion models pre-trained on extensive image-text pairs to enhance the model's cross-modality capabilities. We show that MADM achieves state-of-the-art adaptation performance across various modality tasks, including images to depth, infrared, and event modalities.
  arXiv  Detail & Related papers  (2024-10-29T03:49:40Z)
• Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints [66.63250537475973]
  This paper introduces a diffusion-driven semantic communication framework with advanced VAE-based compression for bandwidth-constrained generative model.<n>Our experimental results demonstrate significant improvements in pixel-level metrics like peak signal to noise ratio (PSNR) and semantic metrics like learned perceptual image patch similarity (LPIPS)
  arXiv  Detail & Related papers  (2024-07-26T02:34:25Z)
• GCD-DDPM: A Generative Change Detection Model Based on Difference-Feature Guided DDPM [7.922421805234563]
  Deep learning methods have recently shown great promise in bitemporal change detection (CD) This work proposes a generative change detection model called GCD-DDPM to directly generate CD maps. Experiments on four high-resolution CD datasets confirm the superior performance of the proposed GCD-DDPM.
  arXiv  Detail & Related papers  (2023-06-06T05:51:50Z)
• Deep Metric Learning for Unsupervised Remote Sensing Change Detection [60.89777029184023]
  Remote Sensing Change Detection (RS-CD) aims to detect relevant changes from Multi-Temporal Remote Sensing Images (MT-RSIs) The performance of existing RS-CD methods is attributed to training on large annotated datasets. This paper proposes an unsupervised CD method based on deep metric learning that can deal with both of these issues.
  arXiv  Detail & Related papers  (2023-03-16T17:52:45Z)
• On Neural Architectures for Deep Learning-based Source Separation of Co-Channel OFDM Signals [104.11663769306566]
  We study the single-channel source separation problem involving frequency-division multiplexing (OFDM) signals. We propose critical domain-informed modifications to the network parameterization, based on insights from OFDM structures.
  arXiv  Detail & Related papers  (2023-03-11T16:29:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.