Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning

• URL: http://arxiv.org/abs/2209.08947v1
• Date: Mon, 19 Sep 2022 11:55:28 GMT
• Title: Age of Semantics in Cooperative Communications: To Expedite Simulation Towards Real via Offline Reinforcement Learning
• Authors: Xianfu Chen and Zhifeng Zhao and Shiwen Mao and Celimuge Wu and Honggang Zhang and Mehdi Bennis
• Abstract summary: We propose the age of semantics (AoS) for measuring semantics freshness of status updates in a cooperative relay communication system. We derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset.
• Score: 53.18060442931179
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The age of information metric fails to correctly describe the intrinsic semantics of a status update. In an intelligent reflecting surface-aided cooperative relay communication system, we propose the age of semantics (AoS) for measuring semantics freshness of the status updates. Specifically, we focus on the status updating from a source node (SN) to the destination, which is formulated as a Markov decision process (MDP). The objective of the SN is to maximize the expected satisfaction of AoS and energy consumption under the maximum transmit power constraint. To seek the optimal control policy, we first derive an online deep actor-critic (DAC) learning scheme under the on-policy temporal difference learning framework. However, implementing the online DAC in practice poses the key challenge in infinitely repeated interactions between the SN and the system, which can be dangerous particularly during the exploration. We then put forward a novel offline DAC scheme, which estimates the optimal control policy from a previously collected dataset without any further interactions with the system. Numerical experiments verify the theoretical results and show that our offline DAC scheme significantly outperforms the online DAC scheme and the most representative baselines in terms of mean utility, demonstrating strong robustness to dataset quality.

Related papers

• Scalable Offline Reinforcement Learning for Mean Field Games [6.8267158622784745]
  Off-MMD is a novel mean-field RL algorithm that approximates equilibrium policies in mean-field games using purely offline data. Our algorithm scales to complex environments and demonstrates strong performance on benchmark tasks like crowd exploration or navigation.
  arXiv  Detail & Related papers  (2024-10-23T14:16:34Z)
• RIS-Based On-the-Air Semantic Communications -- a Diffractional Deep Neural Network Approach [10.626169088908867]
  Current AI-based semantic communication methods require digital hardware for implementation. RIS-based semantic communications offer appealing features, such as light-speed computation, low computational power requirements, and the ability to handle multiple tasks simultaneously.
  arXiv  Detail & Related papers  (2023-12-01T12:15:49Z)
• Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach [58.911515417156174]
  We propose a new definition of Age of Information (AoI) and, based on the redefined AoI, we formulate an online AoI problem for MEC systems. We introduce Post-Decision States (PDSs) to exploit the partial knowledge of the system's dynamics. We also combine PDSs with deep RL to further improve the algorithm's applicability, scalability, and robustness.
  arXiv  Detail & Related papers  (2023-12-01T01:30:49Z)
• Semantic-aware Transmission Scheduling: a Monotonicity-driven Deep Reinforcement Learning Approach [39.681075180578986]
  For cyber-physical systems in the 6G era, semantic communications are required to guarantee application-level performance. In this paper, we first investigate the fundamental properties of the optimal semantic-aware scheduling policy. We then develop advanced deep reinforcement learning (DRL) algorithms by leveraging the theoretical guidelines.
  arXiv  Detail & Related papers  (2023-05-23T05:45:22Z)
• Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach [74.25870052841226]
  A digital twin (DT) leverages a virtual representation of the physical world, along with communication (e.g., 6G), computing, and artificial intelligence (AI) technologies to enable many connected intelligence services. Wireless systems can exploit the paradigm of semantic communication (SC) for facilitating informed decision-making under strict communication constraints. A novel framework called causal semantic communication (CSC) is proposed for DT-based wireless systems.
  arXiv  Detail & Related papers  (2023-04-25T00:15:00Z)
• Active RIS-aided EH-NOMA Networks: A Deep Reinforcement Learning Approach [66.53364438507208]
  An active reconfigurable intelligent surface (RIS)-aided multi-user downlink communication system is investigated. Non-orthogonal multiple access (NOMA) is employed to improve spectral efficiency, and the active RIS is powered by energy harvesting (EH) An advanced LSTM based algorithm is developed to predict users' dynamic communication state. A DDPG based algorithm is proposed to joint control the amplification matrix and phase shift matrix RIS.
  arXiv  Detail & Related papers  (2023-04-11T13:16:28Z)
• Model-based Deep Learning Receiver Design for Rate-Splitting Multiple Access [65.21117658030235]
  This work proposes a novel design for a practical RSMA receiver based on model-based deep learning (MBDL) methods. The MBDL receiver is evaluated in terms of uncoded Symbol Error Rate (SER), throughput performance through Link-Level Simulations (LLS) and average training overhead. Results reveal that the MBDL outperforms by a significant margin the SIC receiver with imperfect CSIR.
  arXiv  Detail & Related papers  (2022-05-02T12:23:55Z)
• Wireless Resource Management in Intelligent Semantic Communication Networks [15.613654766345702]
  We address the user association (UA) and bandwidth allocation problems in an ISC-enabled heterogeneous network (ISC-HetNet) We propose a two-stage solution, including a programming method to obtain an objective, and a algorithm in the second stage to reach the optimality of UA and BA.
  arXiv  Detail & Related papers  (2022-02-15T18:28:28Z)
• A Deep Value-network Based Approach for Multi-Driver Order Dispatching [55.36656442934531]
  We propose a deep reinforcement learning based solution for order dispatching. We conduct large scale online A/B tests on DiDi's ride-dispatching platform. Results show that CVNet consistently outperforms other recently proposed dispatching methods.
  arXiv  Detail & Related papers  (2021-06-08T16:27:04Z)
• Counterfactual Learning of Stochastic Policies with Continuous Actions: from Models to Offline Evaluation [41.21447375318793]
  We introduce a modelling strategy based on a joint kernel embedding of contexts and actions. We empirically show that the optimization aspect of counterfactual learning is important. We propose an evaluation protocol for offline policies in real-world logged systems.
  arXiv  Detail & Related papers  (2020-04-22T07:42:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.