Multi-Modal Sensor Fusion for Proactive Blockage Prediction in mmWave Vehicular Networks

• URL: http://arxiv.org/abs/2507.15769v1
• Date: Mon, 21 Jul 2025 16:25:44 GMT
• Title: Multi-Modal Sensor Fusion for Proactive Blockage Prediction in mmWave Vehicular Networks
• Abstract summary: We propose a proactive blockage prediction framework that utilizes multi-modal sensing, including camera, GPS, LiDAR, and radar inputs.<n>This approach uses modality-specific deep learning models to process each sensor stream independently and fuses their outputs.<n>Our results display the effectiveness and efficiency of multi-modal sensing for mmWave blockage prediction and provide a pathway for proactive wireless communication in dynamic environments.
• Score: 10.4860429348784
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Vehicular communication systems operating in the millimeter wave (mmWave) band are highly susceptible to signal blockage from dynamic obstacles such as vehicles, pedestrians, and infrastructure. To address this challenge, we propose a proactive blockage prediction framework that utilizes multi-modal sensing, including camera, GPS, LiDAR, and radar inputs in an infrastructure-to-vehicle (I2V) setting. This approach uses modality-specific deep learning models to process each sensor stream independently and fuses their outputs using a softmax-weighted ensemble strategy based on validation performance. Our evaluations, for up to 1.5s in advance, show that the camera-only model achieves the best standalone trade-off with an F1-score of 97.1% and an inference time of 89.8ms. A camera+radar configuration further improves accuracy to 97.2% F1 at 95.7ms. Our results display the effectiveness and efficiency of multi-modal sensing for mmWave blockage prediction and provide a pathway for proactive wireless communication in dynamic environments.

Related papers

• A Neural Network Based Teleoperation for Remote Controlled Vehicles [98.38733381483934]
  Direct teleoperation of vehicles faces critical technical bottlenecks.<n> communication latency and the operator's inability to physically perceive unmodeled environmental disturbances.<n>We propose a tailored unilateral teleoperation framework.
  arXiv  Detail & Related papers  (2026-08-11T01:51:08Z)
• LLM-Empowered Multimodal Fusion Framework for Autonomous Driving: Semantic Enhancement and Channel-Adaptive Design [56.76240343283953]
  We propose a Large Language Model-centric Semantic-layer Channel-aware Integrated Perception framework.<n>It fuses a local visual stream with a quality-varying external radar stream used to cover perception-blind spots.<n>Experiments on nuScenes and VIRAT validate our approach.
  arXiv  Detail & Related papers  (2026-07-02T06:44:02Z)
• Edge-Efficient Two-Stream Multimodal Architecture for Non-Intrusive Bathroom Fall Detection [2.669350701173451]
  Falls in wet bathroom environments are a major safety risk for seniors living alone.<n>Recent work has shown that mmWave-only, vibration-only, and existing multimodal schemes can support privacy-preserving, non-intrusive fall detection.<n>We propose a two-stream architecture that encodes radar signals with a Motion--Mamba branch for long-range motion patterns and processes floor vibration.
  arXiv  Detail & Related papers  (2026-03-17T18:54:21Z)
• Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework [1.7834756213254652]
  We present a multi-modal sensing and fusion learning framework as a potential alternative solution to reduce such overheads.<n>In this framework, we first extract the representative features from the sensing modalities by modality specific encoders, then, utilize multi-head cross-modal attention to learn dependencies and correlations between different modalities.<n>The proposed framework achieves up to 96.72% accuracy on predicting top-15 beams correctly, (ii) incurs roughly 0.77 dB average power loss, and (iii) improves the overall latency and beam searching space overheads by 86.81% and 76.56% respectively.
  arXiv  Detail & Related papers  (2026-02-14T05:12:06Z)
• Omnidirectional Solid-State mmWave Radar Perception for UAV Power Line Collision Avoidance [2.707154152696381]
  We present a mmWave radar-based perception system that provides spherical sensing coverage around a small UAV.<n>The system integrates multiple compact solid-state mmWave radar modules to synthesize an omnidirectional field of view.<n>Field experiments on real power lines demonstrate reliable detection at ranges up to 10 m, successful avoidance maneuvers at flight speeds upwards of 10 m/s, and detection of wires as thin as 1.2 mm in diameter.
  arXiv  Detail & Related papers  (2026-02-03T08:02:32Z)
• PRISM: Performer RS-IMLE for Single-pass Multisensory Imitation Learning [51.24484551729328]
  We introduce PRISM, a single-pass policy based on a batch-global rejection-sampling variant of IMLE.<n> PRISM couples a temporal multisensory encoder with a linear-attention generator using a Performer architecture.<n>We demonstrate the efficacy of PRISM on a diverse real-world hardware suite, including loco-manipulation using a Unitree Go2 with a 7-DoF arm D1 and tabletop manipulation with a UR5 manipulator.
  arXiv  Detail & Related papers  (2026-02-02T17:57:37Z)
• A Multimodal Transformer Approach for UAV Detection and Aerial Object Recognition Using Radar, Audio, and Video Data [0.3093890460224435]
  Unmanned aerial vehicle (UAV) detection and aerial object recognition are critical for modern surveillance and security.<n>This research addresses these challenges by designing and rigorously evaluating a novel multimodal Transformer model.<n>It integrates diverse data streams: radar, visual band video (RGB), infrared (IR) video, and audio.
  arXiv  Detail & Related papers  (2025-11-19T10:22:29Z)
• UAV Individual Identification via Distilled RF Fingerprints-Based LLM in ISAC Networks [60.16924915676577]
  Unmanned aerial vehicle (UAV) individual (ID) identification is a critical security surveillance strategy in low-altitude integrated sensing and communication (ISAC) networks.<n>We propose a novel dynamic knowledge distillation (KD)-enabled wireless radio frequency fingerprint large language model (RFF-LLM) framework for UAV ID identification.<n>Experiment results show that the proposed framework achieves 98.38% ID identification accuracy with merely 0.15 million parameters and 2.74 ms response time.
  arXiv  Detail & Related papers  (2025-08-18T03:14:44Z)
• Deep Learning Optimization of Two-State Pinching Antennas Systems [48.70043547158868]
  Pinching antennas (PAs) can dynamically control electromagnetic wave propagation through binary activation states.<n>In this work, we investigate the problem of optimally selecting a subset of fixed-position PAs to activate in a waveguide, when the aim is to maximize the communication rate at a user terminal.
  arXiv  Detail & Related papers  (2025-07-08T17:55:54Z)
• Vision-Based Autonomous MM-Wave Reflector Using ArUco-Driven Angle-of-Arrival Estimation [0.16752458252726457]
  millimeter-wave (mmWave) communication in non-line-of-sight (NLoS) conditions remains a major challenge for military and civilian operations.<n>This paper presents a vision-aided autonomous reflector system designed to enhance mmWave link performance.
  arXiv  Detail & Related papers  (2025-06-05T16:07:22Z)
• Distance Estimation in Outdoor Driving Environments Using Phase-only Correlation Method with Event Cameras [5.690128924544198]
  We present a method for distance estimation using a monocular event camera and a roadside LED bar.<n>The proposed approach achieves over 90% success rate with less than 0.5-meter error for distances ranging from 20 to 60 meters.<n>Future work includes extending this method to full position estimation by leveraging infrastructure such as smart poles equipped with LEDs.
  arXiv  Detail & Related papers  (2025-05-23T07:44:33Z)
• GPS-Aided Deep Learning for Beam Prediction and Tracking in UAV mmWave Communication [6.21540494241516]
  This research presents a GPS-aided deep learning (DL) model that simultaneously predicts current and future optimal beams for UAV mmWave communications.<n>The model reduces overhead by approximately 93% (requiring the training of 2 3 beams instead of 32 beams) with 95% beam prediction accuracy guarantees, and ensures 94% to 96% of predictions exhibit mean power loss not exceeding 1 dB.
  arXiv  Detail & Related papers  (2025-05-23T06:38:00Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• Function Approximation for Reinforcement Learning Controller for Energy from Spread Waves [69.9104427437916]
  Multi-generator Wave Energy Converters (WEC) must handle multiple simultaneous waves coming from different directions called spread waves. These complex devices need controllers with multiple objectives of energy capture efficiency, reduction of structural stress to limit maintenance, and proactive protection against high waves. In this paper, we explore different function approximations for the policy and critic networks in modeling the sequential nature of the system dynamics.
  arXiv  Detail & Related papers  (2024-04-17T02:04:10Z)
• Enhancing Reliability in Federated mmWave Networks: A Practical and Scalable Solution using Radar-Aided Dynamic Blockage Recognition [14.18507067281377]
  This article introduces a new method to improve the dependability of millimeter-wave (mmWave) and terahertz (THz) network services in dynamic outdoor environments. In these settings, line-of-sight (LoS) connections are easily interrupted by moving obstacles like humans and vehicles. The proposed approach, coined as Radar-aided blockage Dynamic Recognition (RaDaR), leverages radar measurements and federated learning (FL) to train a dual-output neural network (NN) model.
  arXiv  Detail & Related papers  (2023-06-22T10:10:25Z)
• StreamYOLO: Real-time Object Detection for Streaming Perception [84.2559631820007]
  We endow the models with the capacity of predicting the future, significantly improving the results for streaming perception. We consider multiple velocities driving scene and propose Velocity-awared streaming AP (VsAP) to jointly evaluate the accuracy. Our simple method achieves the state-of-the-art performance on Argoverse-HD dataset and improves the sAP and VsAP by 4.7% and 8.2% respectively.
  arXiv  Detail & Related papers  (2022-07-21T12:03:02Z)
• Efficient and Robust LiDAR-Based End-to-End Navigation [132.52661670308606]
  We present an efficient and robust LiDAR-based end-to-end navigation framework. We propose Fast-LiDARNet that is based on sparse convolution kernel optimization and hardware-aware model design. We then propose Hybrid Evidential Fusion that directly estimates the uncertainty of the prediction from only a single forward pass.
  arXiv  Detail & Related papers  (2021-05-20T17:52:37Z)
• Deep Learning for Moving Blockage Prediction using Real Millimeter Wave Measurements [18.365889583730507]
  Millimeter wave (mmWave) communication is a key component of 5G and beyond. A sudden blockage in the line of sight link leads to abrupt disconnection, which affects the reliability of the network. We propose a machine learning algorithm learning to predict future blockages by observing what we refer to as the pre-blockage signature.
  arXiv  Detail & Related papers  (2021-01-18T05:34:37Z)
• Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV [119.97450366894718]
  A novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived.
  arXiv  Detail & Related papers  (2020-11-03T16:50:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.