Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios

• URL: http://arxiv.org/abs/2308.06868v1
• Date: Mon, 14 Aug 2023 00:15:01 GMT
• Title: Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios
• Authors: Gouranga Charan, Muhammad Alrabeiah, Tawfik Osman, and Ahmed Alkhateeb
• Abstract summary: This paper extensively investigates the sensing-aided beam prediction problem in a real-world vehicle-to-infrastructure (V2I) scenario. In particular, this paper proposes to utilize visual and positional data to predict the optimal beam indices. The proposed solutions are evaluated on the large-scale real-world DeepSense $6$G dataset.
• Score: 15.287380309115399
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Leveraging sensory information to aid the millimeter-wave (mmWave) and sub-terahertz (sub-THz) beam selection process is attracting increasing interest. This sensory data, captured for example by cameras at the basestations, has the potential of significantly reducing the beam sweeping overhead and enabling highly-mobile applications. The solutions developed so far, however, have mainly considered single-candidate scenarios, i.e., scenarios with a single candidate user in the visual scene, and were evaluated using synthetic datasets. To address these limitations, this paper extensively investigates the sensing-aided beam prediction problem in a real-world multi-object vehicle-to-infrastructure (V2I) scenario and presents a comprehensive machine learning-based framework. In particular, this paper proposes to utilize visual and positional data to predict the optimal beam indices as an alternative to the conventional beam sweeping approaches. For this, a novel user (transmitter) identification solution has been developed, a key step in realizing sensing-aided multi-candidate and multi-user beam prediction solutions. The proposed solutions are evaluated on the large-scale real-world DeepSense $6$G dataset. Experimental results in realistic V2I communication scenarios indicate that the proposed solutions achieve close to $100\%$ top-5 beam prediction accuracy for the scenarios with single-user and close to $95\%$ top-5 beam prediction accuracy for multi-candidate scenarios. Furthermore, the proposed approach can identify the probable transmitting candidate with more than $93\%$ accuracy across the different scenarios. This highlights a promising approach for nearly eliminating the beam training overhead in mmWave/THz communication systems.

Related papers

• OPUS: Occupancy Prediction Using a Sparse Set [64.60854562502523]
  We present a framework to simultaneously predict occupied locations and classes using a set of learnable queries. OPUS incorporates a suite of non-trivial strategies to enhance model performance. Our lightest model achieves superior RayIoU on the Occ3D-nuScenes dataset at near 2x FPS, while our heaviest model surpasses previous best results by 6.1 RayIoU.
  arXiv  Detail & Related papers  (2024-09-14T07:44:22Z)
• RASPNet: A Benchmark Dataset for Radar Adaptive Signal Processing Applications [20.589332431911842]
  The dataset, called RASPNet, consists of 100 realistic scenarios compiled over a variety of topographies and land types from across the contiguous United States. RASPNet intends to fill a prominent gap in the availability of a large-scale, realistic dataset that standardizes the evaluation of adaptive radar processing techniques.
  arXiv  Detail & Related papers  (2024-06-14T00:07:52Z)
• Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery [78.43828998065071]
  Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks. Such pre-training techniques have also been explored recently in the remote sensing domain due to the availability of large amount of unlabelled data. In this paper, we re-visit transformers pre-training and leverage multi-scale information that is effectively utilized with multiple modalities.
  arXiv  Detail & Related papers  (2024-03-08T16:18:04Z)
• Project and Probe: Sample-Efficient Domain Adaptation by Interpolating Orthogonal Features [119.22672589020394]
  We propose a lightweight, sample-efficient approach that learns a diverse set of features and adapts to a target distribution by interpolating these features. Our experiments on four datasets, with multiple distribution shift settings for each, show that Pro$2$ improves performance by 5-15% when given limited target data.
  arXiv  Detail & Related papers  (2023-02-10T18:58:03Z)
• Position Aided Beam Prediction in the Real World: How Useful GPS Locations Actually Are? [12.847571826603726]
  Millimeter-wave (mmWave) communication systems rely on narrow beams for achieving sufficient receive signal power. Adjusting these beams is typically associated with large training overhead. We investigate position-aided beam prediction using a real-world large-scale dataset to derive insights into precisely how much overhead can be saved in practice.
  arXiv  Detail & Related papers  (2022-05-18T16:39:17Z)
• Computer Vision Aided Blockage Prediction in Real-World Millimeter Wave Deployments [11.842197872454848]
  This paper develops a computer vision based solution that processes the visual data captured by a camera installed at the infrastructure node. Based on the adopted real-world dataset, the developed solution achieves $approx 90%$ accuracy in predicting blockages happening within the future.
  arXiv  Detail & Related papers  (2022-03-03T18:38:10Z)
• Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network [8.458980329342799]
  We propose a novel expediting beam selection by leveraging multimodal data collected from sensors like LiDAR, camera images, and GPS. We propose individual and distributed fusion-based deep learning (F-DL) architectures that can execute locally as well as at a mobile edge computing center. Results from extensive evaluations conducted on publicly available synthetic and home-grown real-world datasets reveal 95% and 96% improvement in beam selection speed over classical RF-only beam sweeping.
  arXiv  Detail & Related papers  (2022-01-12T21:55:34Z)
• A Novel Look at LIDAR-aided Data-driven mmWave Beam Selection [24.711393214172148]
  We propose a lightweight neural network (NN) architecture along with the corresponding LIDAR preprocessing. Our NN-based beam selection scheme can achieve 79.9% throughput without any beam search overhead and 95% by searching among as few as 6 beams.
  arXiv  Detail & Related papers  (2021-04-29T18:07:31Z)
• Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups [68.8204255655161]
  We present a method to calibrate the parameters of any pair of sensors involving LiDARs, monocular or stereo cameras. The proposed approach can handle devices with very different resolutions and poses, as usually found in vehicle setups.
  arXiv  Detail & Related papers  (2021-01-12T12:02:26Z)
• AutoSimulate: (Quickly) Learning Synthetic Data Generation [70.82315853981838]
  We propose an efficient alternative for optimal synthetic data generation based on a novel differentiable approximation of the objective. We demonstrate that the proposed method finds the optimal data distribution faster (up to $50times$), with significantly reduced training data generation (up to $30times$) and better accuracy ($+8.7%$) on real-world test datasets than previous methods.
  arXiv  Detail & Related papers  (2020-08-16T11:36:11Z)
• SMART: Simultaneous Multi-Agent Recurrent Trajectory Prediction [72.37440317774556]
  We propose advances that address two key challenges in future trajectory prediction. multimodality in both training data and predictions and constant time inference regardless of number of agents.
  arXiv  Detail & Related papers  (2020-07-26T08:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.