Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network

• URL: http://arxiv.org/abs/2201.04712v1
• Date: Wed, 12 Jan 2022 21:55:34 GMT
• Title: Deep Learning on Multimodal Sensor Data at the Wireless Edge for Vehicular Network
• Authors: Batool Salehi, Guillem Reus-Muns, Debashri Roy, Zifeng Wang, Tong Jian, Jennifer Dy, Stratis Ioannidis, and Kaushik Chowdhury
• Abstract summary: We propose a novel expediting beam selection by leveraging multimodal data collected from sensors like LiDAR, camera images, and GPS. We propose individual and distributed fusion-based deep learning (F-DL) architectures that can execute locally as well as at a mobile edge computing center. Results from extensive evaluations conducted on publicly available synthetic and home-grown real-world datasets reveal 95% and 96% improvement in beam selection speed over classical RF-only beam sweeping.
• Score: 8.458980329342799
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Beam selection for millimeter-wave links in a vehicular scenario is a challenging problem, as an exhaustive search among all candidate beam pairs cannot be assuredly completed within short contact times. We solve this problem via a novel expediting beam selection by leveraging multimodal data collected from sensors like LiDAR, camera images, and GPS. We propose individual modality and distributed fusion-based deep learning (F-DL) architectures that can execute locally as well as at a mobile edge computing center (MEC), with a study on associated tradeoffs. We also formulate and solve an optimization problem that considers practical beam-searching, MEC processing and sensor-to-MEC data delivery latency overheads for determining the output dimensions of the above F-DL architectures. Results from extensive evaluations conducted on publicly available synthetic and home-grown real-world datasets reveal 95% and 96% improvement in beam selection speed over classical RF-only beam sweeping, respectively. F-DL also outperforms the state-of-the-art techniques by 20-22% in predicting top-10 best beam pairs.

Related papers

• Hyperdimensional Computing Empowered Federated Foundation Model over Wireless Networks for Metaverse [56.384390765357004]
  We propose an integrated federated split learning and hyperdimensional computing framework for emerging foundation models. This novel approach reduces communication costs, computation load, and privacy risks, making it suitable for resource-constrained edge devices in the Metaverse.
  arXiv  Detail & Related papers  (2024-08-26T17:03:14Z)
• Semi-Federated Learning: Convergence Analysis and Optimization of A Hybrid Learning Framework [70.83511997272457]
  We propose a semi-federated learning (SemiFL) paradigm to leverage both the base station (BS) and devices for a hybrid implementation of centralized learning (CL) and FL. We propose a two-stage algorithm to solve this intractable problem, in which we provide the closed-form solutions to the beamformers.
  arXiv  Detail & Related papers  (2023-10-04T03:32:39Z)
• Multi-Modal Multi-Task (3MT) Road Segmentation [0.8287206589886879]
  We focus on using raw sensor inputs instead of, as it is typically done in many SOTA works, leveraging architectures that require high pre-processing costs. This study presents a cost-effective and highly accurate solution for road segmentation by integrating data from multiple sensors within a multi-task learning architecture.
  arXiv  Detail & Related papers  (2023-08-23T08:15:15Z)
• Camera Based mmWave Beam Prediction: Towards Multi-Candidate Real-World Scenarios [15.287380309115399]
  This paper extensively investigates the sensing-aided beam prediction problem in a real-world vehicle-to-infrastructure (V2I) scenario. In particular, this paper proposes to utilize visual and positional data to predict the optimal beam indices. The proposed solutions are evaluated on the large-scale real-world DeepSense $6$G dataset.
  arXiv  Detail & Related papers  (2023-08-14T00:15:01Z)
• Deep Learning and Image Super-Resolution-Guided Beam and Power Allocation for mmWave Networks [80.37827344656048]
  We develop a deep learning (DL)-guided hybrid beam and power allocation approach for millimeter-wave (mmWave) networks. We exploit the synergy of supervised learning and super-resolution technology to enable low-overhead beam- and power allocation.
  arXiv  Detail & Related papers  (2023-05-08T05:40:54Z)
• Fast Beam Alignment via Pure Exploration in Multi-armed Bandits [91.11360914335384]
  We develop a bandit-based fast BA algorithm to reduce BA latency for millimeter-wave (mmWave) communications. Our algorithm is named Two-Phase Heteroscedastic Track-and-Stop (2PHT&S)
  arXiv  Detail & Related papers  (2022-10-23T05:57:39Z)
• Online Data Selection for Federated Learning with Limited Storage [53.46789303416799]
  Federated Learning (FL) has been proposed to achieve distributed machine learning among networked devices. The impact of on-device storage on the performance of FL is still not explored. In this work, we take the first step to consider the online data selection for FL with limited on-device storage.
  arXiv  Detail & Related papers  (2022-09-01T03:27:33Z)
• Federated Learning for Energy-limited Wireless Networks: A Partial Model Aggregation Approach [79.59560136273917]
  limited communication resources, bandwidth and energy, and data heterogeneity across devices are main bottlenecks for federated learning (FL) We first devise a novel FL framework with partial model aggregation (PMA) The proposed PMA-FL improves 2.72% and 11.6% accuracy on two typical heterogeneous datasets.
  arXiv  Detail & Related papers  (2022-04-20T19:09:52Z)
• A Novel Look at LIDAR-aided Data-driven mmWave Beam Selection [24.711393214172148]
  We propose a lightweight neural network (NN) architecture along with the corresponding LIDAR preprocessing. Our NN-based beam selection scheme can achieve 79.9% throughput without any beam search overhead and 95% by searching among as few as 6 beams.
  arXiv  Detail & Related papers  (2021-04-29T18:07:31Z)
• Federated mmWave Beam Selection Utilizing LIDAR Data [3.7352534957395522]
  We propose distributed LIDAR aided beam selection for V2I mmWave communication systems utilizing federated training. In the proposed scheme, connected vehicles collaborate to train a shared neural network (NN) on their locally available LIDAR data during normal operation of the system. We also propose an alternative reduced-complexity convolutional NN (CNN) architecture and LIDAR preprocessing, which significantly outperforms previous works in terms of both the performance and the complexity.
  arXiv  Detail & Related papers  (2021-02-04T18:49:20Z)
• SpaceNet 6: Multi-Sensor All Weather Mapping Dataset [13.715388432549373]
  We present an open Multi-Sensor All Weather Mapping (MSAW) dataset and challenge. MSAW covers 120 km2 over multiple overlapping collects and is annotated with over 48,000 unique building footprints labels. We present a baseline and benchmark for building footprint extraction with SAR data and find that state-of-the-art segmentation models pre-trained on optical data, and then trained on SAR.
  arXiv  Detail & Related papers  (2020-04-14T13:43:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.