Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity

• URL: http://arxiv.org/abs/2605.05071v1
• Date: Wed, 06 May 2026 16:07:26 GMT
• Title: Look Once, Beam Twice: Camera-Primed Real-Time Double-Directional mmWave Beam Management for Vehicular Connectivity
• Abstract summary: Millimeter-wave (mmWave) promise multi-gigabit connectivity for vehicle-to-everything (V2X) networks.<n> Reliable V2X connectivity requires fast, double-directional beam alignment.<n>This paper presents VIsion-based BEamforming(VIBE), a hybrid model-based, closed-loop, learning architecture for real-time mmWave beam management.
• Score: 4.279449367355336
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Millimeter-wave (mmWave) frequencies promise multi-gigabit connectivity for vehicle-to-everything (V2X) networks, but face challenges in terms of severe path loss and mobility-related beam misalignment. Reliable V2X connectivity requires fast, double-directional beam alignment. However, existing methods suffer from high training overhead and limited generalization to unseen scenarios. This paper presents VIsion-based BEamforming(VIBE), a hybrid model-based, closed-loop, learning architecture for real-time double-directional mmWave beam management primed by camera sensing. VIBE fuses machine learning, model-based reasoning, and closed-loop RF feedback to balance beam-pair establishment latency with link quality. VIBE bypasses exhaustive training overhead and accelerates link establishment by leveraging camera observations to reduce the beam-search space. Lightweight beam refinement and offset tracking mechanisms adaptively refine beams in response to dynamic application requirements. VIBE is implemented and evaluated across online indoor/outdoor testbeds, public datasets, and real-time vehicular experiments, demonstrating strong generalization capabilities, making it suitable for real-time V2X communication. Comparisons with 5G NR hierarchical beamforming show that VIBE consistently maintains lower outage rates. Furthermore, VIBE outperforms state-of-the-art end-to-end ML models for beam selection when evaluated on public datasets and achieves outage rates as low as 1.1-1.4 %. The results show that a hybrid model-based, closed-loop learning architecture is better suited for real-world mmWave vehicular connectivity than end-to-end trained ML models. For reproducibility, we publish our code to https://github.com/UNL-CPN-Lab/Look-Once-Beam-Twice.

Related papers

• A Neural Network Based Teleoperation for Remote Controlled Vehicles [98.38733381483934]
  Direct teleoperation of vehicles faces critical technical bottlenecks.<n> communication latency and the operator's inability to physically perceive unmodeled environmental disturbances.<n>We propose a tailored unilateral teleoperation framework.
  arXiv  Detail & Related papers  (2026-08-11T01:51:08Z)
• Adaptive Cross-Modal Fusion with Sparse Attention for Pedestrian Crossing Intention Prediction [4.8915598762204135]
  ADAPT (Adaptive Domain-Aware Pedestrian Crossing Transformer) is a multimodal framework for predicting pedestrian crossing intention.<n> ADAPT processes four aligned visual modalities, including RGB images, local depth maps, global semantic maps, and global depth maps.<n>Experiments on the JAAD and PIE benchmark datasets demonstrate that ADAPT consistently outperforms existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2026-07-14T03:05:09Z)
• Multi-Modal Sensing and Fusion in mmWave Beamforming for Connected Vehicles: A Transformer Based Framework [1.7834756213254652]
  We present a multi-modal sensing and fusion learning framework as a potential alternative solution to reduce such overheads.<n>In this framework, we first extract the representative features from the sensing modalities by modality specific encoders, then, utilize multi-head cross-modal attention to learn dependencies and correlations between different modalities.<n>The proposed framework achieves up to 96.72% accuracy on predicting top-15 beams correctly, (ii) incurs roughly 0.77 dB average power loss, and (iii) improves the overall latency and beam searching space overheads by 86.81% and 76.56% respectively.
  arXiv  Detail & Related papers  (2026-02-14T05:12:06Z)
• Neural Beamforming with Doppler-Aware Sparse Attention for High Mobility Environments [35.90167049594104]
  Deep learning-based beamforming offers an alternative with nonlinear mappings from channel state information to beamforming weights.<n> Transformer-based models are particularly effective due to their ability to model long-range dependencies across time and frequency.<n>Recent studies address this issue through sparse attention mechanisms that reduce complexity while maintaining, yet often employ patterns that disregard channel dynamics.
  arXiv  Detail & Related papers  (2025-11-05T16:51:26Z)
• CAR-BRAINet: Sub-6GHz Aided Spatial Adaptive Beam Prediction with Multi Head Attention for Heterogeneous Vehicular Networks [4.84929109771831]
  Heterogeneous Vehicular Networks (HetVNets) play a key role by stacking different communication technologies such as sub-6GHz, mm-wave and DSRC to meet diverse connectivity needs of 5G/B5G vehicular networks.<n>HetVNet helps address the humongous user demands-but maintaining a steady connection in a highly mobile, real-world conditions remain a challenge.<n>This paper introduces a lightweight deep learning-based solution termed-"CAR-BRAINet" which consists of convolutional neural networks with a powerful multi-head attention mechanism.
  arXiv  Detail & Related papers  (2025-09-02T05:17:23Z)
• Learning Frequency and Memory-Aware Prompts for Multi-Modal Object Tracking [74.15663758681849]
  We present Learning Frequency and Memory-Aware Prompts, a dual-adapter framework that injects lightweight prompts into a frozen RGB tracker.<n>A frequency-guided visual adapter adaptively transfers complementary cues across modalities.<n>A multilevel memory adapter with short, long, and permanent memory stores, updates, and retrieves reliable temporal context.
  arXiv  Detail & Related papers  (2025-06-30T15:38:26Z)
• FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systems [51.15230303652732]
  FLEX (F Low EXpert) is a backbone architecture for generative modeling of-temporal physical systems.<n>It reduces the variance of the velocity field in the diffusion model, which helps stabilize training.<n>It achieves accurate predictions for super-resolution and forecasting tasks using as few features as two reverse diffusion steps.
  arXiv  Detail & Related papers  (2025-05-23T00:07:59Z)
• World Model-Based Learning for Long-Term Age of Information Minimization in Vehicular Networks [53.98633183204453]
  In this paper, a novel world model-based learning framework is proposed to minimize packet-completeness-aware age of information (CAoI) in a vehicular network.<n>A world model framework is proposed to jointly learn a dynamic model of the mmWave V2X environment and use it to imagine trajectories for learning how to perform link scheduling.<n>In particular, the long-term policy is learned in differentiable imagined trajectories instead of environment interactions.
  arXiv  Detail & Related papers  (2025-05-03T06:23:18Z)
• Resource-Efficient Beam Prediction in mmWave Communications with Multimodal Realistic Simulation Framework [57.994965436344195]
  Beamforming is a key technology in millimeter-wave (mmWave) communications that improves signal transmission by optimizing directionality and intensity.<n> multimodal sensing-aided beam prediction has gained significant attention, using various sensing data to predict user locations or network conditions.<n>Despite its promising potential, the adoption of multimodal sensing-aided beam prediction is hindered by high computational complexity, high costs, and limited datasets.
  arXiv  Detail & Related papers  (2025-04-07T15:38:25Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• Self-Organizing mmWave MIMO Cell-Free Networks With Hybrid Beamforming: A Hierarchical DRL-Based Design [30.70798412145064]
  In a cell-free wireless network, distributed access points (APs) jointly serve all user equipments (UEs) within the their coverage area by using the same time/frequency resources. We propose several network partitioning based on deep learning (DRL) To design interference between different cell-freeworks, we develop a novel hybrid beamst-digital beam model.
  arXiv  Detail & Related papers  (2021-03-17T03:31:52Z)
• Terahertz-Band Joint Ultra-Massive MIMO Radar-Communications: Model-Based and Model-Free Hybrid Beamforming [45.257328085051974]
  Wireless communications and sensing at terahertz (THz) band are investigated as promising short-range technologies. Ultra-massive multiple-input multiple-output (UM-MIMO) antenna systems have been proposed for THz communications to compensate propagation losses. We develop THz hybrid beamformers based on both model-based and model-free techniques for a new group-of-subarrays (GoSA) UM-MIMO structure.
  arXiv  Detail & Related papers  (2021-02-27T21:28:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.