Robust Beam Prediction for V2X Networks with Multi-Modal Sensing

• URL: http://arxiv.org/abs/2609.10200v1
• Date: Wed, 09 Sep 2026 14:04:55 GMT
• Title: Robust Beam Prediction for V2X Networks with Multi-Modal Sensing
• Abstract summary: Integrated sensing and communication (ISAC) provides a promising foundation for beam prediction in future vehicle-to-everything (V2X) networks.<n>This paper proposes a multi-modal beam prediction framework for V2X networks.<n> Specifically, we develop BeamTransFuser, a hierarchical Transformer-based architecture that fuses camera, LiDAR, radar, and GPS observations for robust beam prediction.
• Score: 20.892529341998877
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Integrated sensing and communication (ISAC) provides a promising foundation for beam prediction in future vehicle-to-everything (V2X) networks. However, existing sensing-assisted beamforming methods still rely heavily on radio-frequency sensing, which may become unreliable in complex vehicular environments. Meanwhile, the growing availability of heterogeneous sensors, such as cameras and LiDAR, offers new opportunities to improve beam prediction through richer environmental perception. Motivated by this, this paper proposes a multi-modal beam prediction framework for V2X networks. Specifically, we develop BeamTransFuser, a hierarchical Transformer-based architecture that progressively fuses camera, LiDAR, radar, and GPS observations for robust beam prediction. In addition, to handle possible missing modalities in practical deployment, we introduce a generative module that reconstructs missing modality features from the available observations. Experimental results on a real-world multi-modal V2X dataset show that the proposed framework consistently outperforms representative baselines, while the generative module further improves robustness under incomplete sensing conditions.

Related papers

• BeamRMX: Radiation-Pattern-Driven Learning for Generalizable Beam Radio Map Prediction and Beam Management [26.316069202754473]
  6G wireless networks are driving larger antenna arrays and highly directional multi-beam transmission.<n>BeamRMX is the first dedicated framework to treat the spatial radiation pattern as the primary BeamRM query.<n>XBase learns multiscale interactions between the radiation query and scene geometry.
  arXiv  Detail & Related papers  (2026-09-01T02:59:08Z)
• Learning Spatial Structure from Pre-Beamforming Per-Antenna Range-Doppler Radar Data via Visibility-Aware Cross-Modal Supervision [2.348805691644085]
  Can meaningful spatial structure be learned directly from pre-beamforming per-antenna range-Doppler measurements?<n>Experiments are conducted on a 6-TX x 8-RX (48 virtual antennas)<n>We operate on pre-beamforming per-antenna RD tensors using a dual-chirp shared-weight encoder trained in an end-to-end, fully data-driven manner.
  arXiv  Detail & Related papers  (2026-04-02T11:39:00Z)
• Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework [1.7834756213254652]
  We present a multi-modal sensing and fusion learning framework as a potential alternative solution to reduce such overheads.<n>In this framework, we first extract the representative features from the sensing modalities by modality specific encoders, then, utilize multi-head cross-modal attention to learn dependencies and correlations between different modalities.<n>The proposed framework achieves up to 96.72% accuracy on predicting top-15 beams correctly, (ii) incurs roughly 0.77 dB average power loss, and (iii) improves the overall latency and beam searching space overheads by 86.81% and 76.56% respectively.
  arXiv  Detail & Related papers  (2026-02-14T05:12:06Z)
• RadarGen: Automotive Radar Point Cloud Generation from Cameras [64.69976771710057]
  We present RadarGen, a diffusion model for synthesizing realistic automotive radar point clouds from multi-view camera imagery.<n>RadarGen adapts efficient image-latent diffusion to the radar domain by representing radar measurements in bird's-eye-view form.<n>We show that RadarGen captures characteristic radar measurement distributions and reduces the gap to perception models trained on real data.
  arXiv  Detail & Related papers  (2025-12-19T18:57:33Z)
• Multi-Modal Sensing Aided mmWave Beamforming for V2V Communications with Transformers [1.9483189922830135]
  We present a multi-modal sensing and fusion learning framework as a potential alternative solution to reduce such overheads.<n>In this framework, we first extract the features individually from the visual and GPS coordinates sensing modalities by modality specific encoders.<n>We then fuse the multimodal features to obtain predicted top-k beams so that the best line-of-sight links can be proactively established.
  arXiv  Detail & Related papers  (2025-09-14T06:03:42Z)
• M2BeamLLM: Multimodal Sensing-empowered mmWave Beam Prediction with Large Language Models [22.009889991924453]
  M2BeamLLM integrates multi-modal sensor data, including images, radar, LiDAR, and GPS.<n>Its prediction performance consistently improves with increased diversity in sensing modalities.<n>Our study provides an efficient and intelligent beam prediction solution for vehicle-to-infrastructure (V2I) mmWave communication systems.
  arXiv  Detail & Related papers  (2025-06-17T13:58:36Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• BEVWorld: A Multimodal World Simulator for Autonomous Driving via Scene-Level BEV Latents [56.33989853438012]
  We propose BEVWorld, a framework that transforms multimodal sensor inputs into a unified and compact Bird's Eye View latent space for holistic environment modeling.<n>The proposed world model consists of two main components: a multi-modal tokenizer and a latent BEV sequence diffusion model.
  arXiv  Detail & Related papers  (2024-07-08T07:26:08Z)
• Joint Sensing, Communication, and AI: A Trifecta for Resilient THz User Experiences [118.91584633024907]
  A novel joint sensing, communication, and artificial intelligence (AI) framework is proposed so as to optimize extended reality (XR) experiences over terahertz (THz) wireless systems.
  arXiv  Detail & Related papers  (2023-04-29T00:39:50Z)
• Fully Convolutional One-Stage 3D Object Detection on LiDAR Range Images [96.66271207089096]
  FCOS-LiDAR is a fully convolutional one-stage 3D object detector for LiDAR point clouds of autonomous driving scenes. We show that an RV-based 3D detector with standard 2D convolutions alone can achieve comparable performance to state-of-the-art BEV-based detectors.
  arXiv  Detail & Related papers  (2022-05-27T05:42:16Z)
• V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer [58.71845618090022]
  We build a holistic attention model, namely V2X-ViT, to fuse information across on-road agents. V2X-ViT consists of alternating layers of heterogeneous multi-agent self-attention and multi-scale window self-attention. To validate our approach, we create a large-scale V2X perception dataset.
  arXiv  Detail & Related papers  (2022-03-20T20:18:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.