Reinforcement Learning for Minimizing Age of Information in Real-time Internet of Things Systems with Realistic Physical Dynamics

• URL: http://arxiv.org/abs/2104.01527v1
• Date: Sun, 4 Apr 2021 03:17:26 GMT
• Title: Reinforcement Learning for Minimizing Age of Information in Real-time Internet of Things Systems with Realistic Physical Dynamics
• Authors: Sihua Wang, Mingzhe Chen, Zhaohui Yang, Changchuan Yin, Walid Saad, Shuguang Cui, H. Vincent Poor
• Abstract summary: This paper studies the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices. A distributed reinforcement learning approach is proposed to optimize the sampling policy. Simulations with real data of PM 2.5 pollution show that the proposed algorithm can reduce the sum of AoI by up to 17.8% and 33.9%.
• Score: 158.67956699843168
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices is studied. In the considered model, each IoT device monitors a physical process that follows nonlinear dynamics. As the dynamics of the physical process vary over time, each device must find an optimal sampling frequency to sample the real-time dynamics of the physical system and send sampled information to a base station (BS). Due to limited wireless resources, the BS can only select a subset of devices to transmit their sampled information. Meanwhile, changing the sampling frequency will also impact the energy used by each device for sampling and information transmission. Thus, it is necessary to jointly optimize the sampling policy of each device and the device selection scheme of the BS so as to accurately monitor the dynamics of the physical process using minimum energy. This problem is formulated as an optimization problem whose goal is to minimize the weighted sum of AoI cost and energy consumption. To solve this problem, a distributed reinforcement learning approach is proposed to optimize the sampling policy. The proposed learning method enables the IoT devices to find the optimal sampling policy using their local observations. Given the sampling policy, the device selection scheme can be optimized so as to minimize the weighted sum of AoI and energy consumption of all devices. Simulations with real data of PM 2.5 pollution show that the proposed algorithm can reduce the sum of AoI by up to 17.8% and 33.9% and the total energy consumption by up to 13.2% and 35.1%, compared to a conventional deep Q network method and a uniform sampling policy.

Related papers

• Automatic AI Model Selection for Wireless Systems: Online Learning via Digital Twinning [50.332027356848094]
  AI-based applications are deployed at intelligent controllers to carry out functionalities like scheduling or power control. The mapping between context and AI model parameters is ideally done in a zero-shot fashion. This paper introduces a general methodology for the online optimization of AMS mappings.
  arXiv  Detail & Related papers  (2024-06-22T11:17:50Z)
• Iterated Denoising Energy Matching for Sampling from Boltzmann Densities [109.23137009609519]
  Iterated Denoising Energy Matching (iDEM) iDEM alternates between (I) sampling regions of high model density from a diffusion-based sampler and (II) using these samples in our matching objective. We show that the proposed approach achieves state-of-the-art performance on all metrics and trains $2-5times$ faster.
  arXiv  Detail & Related papers  (2024-02-09T01:11:23Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• Sample-Driven Federated Learning for Energy-Efficient and Real-Time IoT Sensing [22.968661040226756]
  We introduce an online reinforcement learning algorithm named Sample-driven Control for Federated Learning (SCFL) built on the Soft Actor-Critic (A2C) framework. SCFL enables the agent to dynamically adapt and find the global optima even in changing environments.
  arXiv  Detail & Related papers  (2023-10-11T13:50:28Z)
• Deep Reinforcement Learning Based Power Allocation for Minimizing AoI and Energy Consumption in MIMO-NOMA IoT Systems [27.19355345123451]
  Multi-input multi-out and non-orthogonal multiple access (MIMO-NOMA) internet-of-things (IoT) systems can improve channel capacity and spectrum efficiency distinctly to support the real-time applications. Age of information (AoI) is an important metric for real-time application, but there is no literature have minimized AoI of the. MIMO-NOMA IoT system, where the transmission rate is not a constant in the SIC process and the noise is in the. MIMO-NOMA IoT system based on deep reinforcement learning (DRL)
  arXiv  Detail & Related papers  (2023-03-11T14:09:46Z)
• Resource Allocation of Federated Learning for the Metaverse with Mobile Augmented Reality [13.954907748381743]
  Metaverse applications via mobile augmented reality (MAR) require rapid and accurate object detection to mix digital data with the real world. Federated learning (FL) is an intriguing distributed machine learning approach due to its privacy-preserving characteristics. We formulate an optimization problem to minimize a weighted combination of total energy consumption, completion time and model accuracy.
  arXiv  Detail & Related papers  (2022-11-16T06:37:32Z)
• Joint Optimization of Energy Consumption and Completion Time in Federated Learning [16.127019859725785]
  Federated Learning (FL) is an intriguing distributed machine learning approach due to its privacy-preserving characteristics. We formulate an algorithm to balance the trade-off between energy and execution latency, and thus accommodate different demands and application scenarios.
  arXiv  Detail & Related papers  (2022-09-29T16:05:28Z)
• Federated Learning for Energy-limited Wireless Networks: A Partial Model Aggregation Approach [79.59560136273917]
  limited communication resources, bandwidth and energy, and data heterogeneity across devices are main bottlenecks for federated learning (FL) We first devise a novel FL framework with partial model aggregation (PMA) The proposed PMA-FL improves 2.72% and 11.6% accuracy on two typical heterogeneous datasets.
  arXiv  Detail & Related papers  (2022-04-20T19:09:52Z)
• Fine-Grained Data Selection for Improved Energy Efficiency of Federated Edge Learning [2.924868086534434]
  In Federated edge learning (FEEL), energy-constrained devices at the network edge consume significant energy when training and uploading their local machine learning models. This work proposes novel solutions for energy-efficient FEEL by jointly considering local training data, available computation, and communications resources.
  arXiv  Detail & Related papers  (2021-06-20T10:51:32Z)
• Multi-Agent Meta-Reinforcement Learning for Self-Powered and Sustainable Edge Computing Systems [87.4519172058185]
  An effective energy dispatch mechanism for self-powered wireless networks with edge computing capabilities is studied. A novel multi-agent meta-reinforcement learning (MAMRL) framework is proposed to solve the formulated problem. Experimental results show that the proposed MAMRL model can reduce up to 11% non-renewable energy usage and by 22.4% the energy cost.
  arXiv  Detail & Related papers  (2020-02-20T04:58:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.