Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates

• URL: http://arxiv.org/abs/2412.18500v1
• Date: Tue, 24 Dec 2024 15:32:58 GMT
• Title: Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates
• Authors: Mamady Delamou, Ahmed Naeem, Huseyin Arslan, El Mehdi Amhoud,
• Abstract summary: Millimeter wave (mmWave)-based frequency-division multiplexing (OFDM) stands out as a suitable alternative for high-resolution sensing and high-speed data transmission. In this work, we consider an autonomous vehicle network where an AV utilizes its queue state information (QSI) and channel state information (CSI) in conjunction with reinforcement learning techniques to manage communication and sensing.
• Score: 2.607046313483251
• License:
• Abstract: Millimeter wave (mmWave)-based orthogonal frequency-division multiplexing (OFDM) stands out as a suitable alternative for high-resolution sensing and high-speed data transmission. To meet communication and sensing requirements, many works propose a static configuration where the wave's hyperparameters such as the number of symbols in a frame and the number of frames in a communication slot are already predefined. However, two facts oblige us to redefine the problem, (1) the environment is often dynamic and uncertain, and (2) mmWave is severely impacted by wireless environments. A striking example where this challenge is very prominent is autonomous vehicle (AV). Such a system leverages integrated sensing and communication (ISAC) using mmWave to manage data transmission and the dynamism of the environment. In this work, we consider an autonomous vehicle network where an AV utilizes its queue state information (QSI) and channel state information (CSI) in conjunction with reinforcement learning techniques to manage communication and sensing. This enables the AV to achieve two primary objectives: establishing a stable communication link with other AVs and accurately estimating the velocities of surrounding objects with high resolution. The communication performance is therefore evaluated based on the queue state, the effective data rate, and the discarded packets rate. In contrast, the effectiveness of the sensing is assessed using the velocity resolution. In addition, we exploit adaptive OFDM techniques for dynamic modulation, and we suggest a reward function that leverages the age of updates to handle the communication buffer and improve sensing. The system is validated using advantage actor-critic (A2C) and proximal policy optimization (PPO). Furthermore, we compare our solution with the existing design and demonstrate its superior performance by computer simulations.

Related papers

• Deep Reinforcement Learning-Based User Scheduling for Collaborative Perception [24.300126250046894]
  Collaborative perception is envisioned to improve perceptual accuracy by using vehicle-to-everything (V2X) communication. Due to limited communication resources, it is impractical for all units to transmit sensing data such as point clouds or high-definition video. We propose a deep reinforcement learning-based V2X user scheduling algorithm for collaborative perception.
  arXiv  Detail & Related papers  (2025-02-12T04:45:00Z)
• Aerial Reliable Collaborative Communications for Terrestrial Mobile Users via Evolutionary Multi-Objective Deep Reinforcement Learning [59.660724802286865]
  Unmanned aerial vehicles (UAVs) have emerged as the potential aerial base stations (BSs) to improve terrestrial communications. This work employs collaborative beamforming through a UAV-enabled virtual antenna array to improve transmission performance from the UAV to terrestrial mobile users.
  arXiv  Detail & Related papers  (2025-02-09T09:15:47Z)
• Semantic Communication for Cooperative Perception using HARQ [51.148203799109304]
  We leverage an importance map to distill critical semantic information, introducing a cooperative perception semantic communication framework. To counter the challenges posed by time-varying multipath fading, our approach incorporates the use of frequency-division multiplexing (OFDM) along with channel estimation and equalization strategies. We introduce a novel semantic error detection method that is integrated with our semantic communication framework in the spirit of hybrid automatic repeated request (HARQ)
  arXiv  Detail & Related papers  (2024-08-29T08:53:26Z)
• Improved Q-learning based Multi-hop Routing for UAV-Assisted Communication [4.799822253865053]
  This paper proposes a novel, Improved Q-learning-based Multi-hop Routing (IQMR) algorithm for optimal UAV-assisted communication systems. Using Q(lambda) learning for routing decisions, IQMR substantially enhances energy efficiency and network data throughput.
  arXiv  Detail & Related papers  (2024-08-17T06:24:31Z)
• Deep-Reinforcement-Learning-Based AoI-Aware Resource Allocation for RIS-Aided IoV Networks [43.443526528832145]
  We propose a RIS-assisted internet of vehicles (IoV) network, considering the vehicle-to-everything (V2X) communication method. In order to improve the timeliness of vehicle-to-infrastructure (V2I) links and the stability of vehicle-to-vehicle (V2V) links, we introduce the age of information (AoI) model and the payload transmission probability model.
  arXiv  Detail & Related papers  (2024-06-17T06:16:07Z)
• Semantic Communication Enabling Robust Edge Intelligence for Time-Critical IoT Applications [87.05763097471487]
  This paper aims to design robust Edge Intelligence using semantic communication for time-critical IoT applications. We analyze the effect of image DCT coefficients on inference accuracy and propose the channel-agnostic effectiveness encoding for offloading.
  arXiv  Detail & Related papers  (2022-11-24T20:13:17Z)
• Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning [53.18060442931179]
  We propose the age of semantics (AoS) for measuring semantics freshness of status updates in a cooperative relay communication system. We derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset.
  arXiv  Detail & Related papers  (2022-09-19T11:55:28Z)
• Reinforcement Learning for Joint V2I Network Selection and Autonomous Driving Policies [14.518558523319518]
  Vehicle-to-Infrastructure (V2I) communication is becoming critical for the enhanced reliability of autonomous vehicles (AVs) It is critical to simultaneously optimize the AVs' network selection and driving policies in order to minimize road collisions. We develop a reinforcement learning framework to characterize efficient network selection and autonomous driving policies.
  arXiv  Detail & Related papers  (2022-08-03T04:33:02Z)
• Rethinking the Tradeoff in Integrated Sensing and Communication: Recognition Accuracy versus Communication Rate [21.149708253108788]
  Integrated sensing and communication (ISAC) is a promising technology to improve the band-utilization efficiency. There exists a tradeoff between the sensing and communication performance. This paper formulates and solves a multi-objective optimization problem which simultaneously maximizes the recognition accuracy and the communication data rate.
  arXiv  Detail & Related papers  (2021-07-20T17:00:35Z)
• Path Design and Resource Management for NOMA enhanced Indoor Intelligent Robots [58.980293789967575]
  A communication enabled indoor intelligent robots (IRs) service framework is proposed. Lego modeling method is proposed, which can deterministically describe the indoor layout and channel state. The investigated radio map is invoked as a virtual environment to train the reinforcement learning agent.
  arXiv  Detail & Related papers  (2020-11-23T21:45:01Z)
• Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV [119.97450366894718]
  A novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived.
  arXiv  Detail & Related papers  (2020-11-03T16:50:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.