Measuring and Mitigating Interference in Reinforcement Learning

• URL: http://arxiv.org/abs/2307.04887v1
• Date: Mon, 10 Jul 2023 20:20:20 GMT
• Title: Measuring and Mitigating Interference in Reinforcement Learning
• Authors: Vincent Liu, Han Wang, Ruo Yu Tao, Khurram Javed, Adam White, Martha White
• Abstract summary: Catastrophic interference is common in many network-based learning systems. We provide a definition and novel measure of interference for value-based reinforcement learning methods.
• Score: 30.38857177546063
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Catastrophic interference is common in many network-based learning systems, and many proposals exist for mitigating it. Before overcoming interference we must understand it better. In this work, we provide a definition and novel measure of interference for value-based reinforcement learning methods such as Fitted Q-Iteration and DQN. We systematically evaluate our measure of interference, showing that it correlates with instability in control performance, across a variety of network architectures. Our new interference measure allows us to ask novel scientific questions about commonly used deep learning architectures and study learning algorithms which mitigate interference. Lastly, we outline a class of algorithms which we call online-aware that are designed to mitigate interference, and show they do reduce interference according to our measure and that they improve stability and performance in several classic control environments.

Related papers

• Physically Parameterized Differentiable MUSIC for DoA Estimation with Uncalibrated Arrays [18.68871336059738]
  Direction of arrival (DoA) estimation is a common sensing problem in radar, sonar, audio, and wireless communication systems. This study introduces a joint DoA estimation and hardware impairment learning scheme following a model-based approach.
  arXiv  Detail & Related papers  (2024-11-06T09:14:26Z)
• A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing [4.000213034401085]
  We introduce "causal network motifs" and utilize transparent machine learning models to characterize network interference patterns. Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners.
  arXiv  Detail & Related papers  (2023-08-18T19:37:55Z)
• Edge Intelligence Over the Air: Two Faces of Interference in Federated Learning [95.31679010587473]
  Federated edge learning is envisioned as the bedrock of enabling intelligence in next-generation wireless networks. This article provides a comprehensive overview of the positive and negative effects of interference on over-the-air-based edge learning systems.
  arXiv  Detail & Related papers  (2023-06-17T09:04:48Z)
• Deep Reinforcement Learning for Interference Management in UAV-based 3D Networks: Potentials and Challenges [137.47736805685457]
  We show that interference can still be effectively mitigated even without knowing its channel information. By harnessing interference, the proposed solutions enable the continued growth of civilian UAVs.
  arXiv  Detail & Related papers  (2023-05-11T18:06:46Z)
• Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
  In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions. For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions. The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
  arXiv  Detail & Related papers  (2022-05-13T21:09:41Z)
• Interference Suppression Using Deep Learning: Current Approaches and Open Challenges [2.179313476241343]
  In this paper, we review a wide range of techniques that have used deep learning to suppress interference. We provide comparison and guidelines for many different types of deep learning techniques in interference suppression. In addition, we highlight challenges and potential future research directions for the successful adoption of deep learning in interference suppression.
  arXiv  Detail & Related papers  (2021-12-16T16:07:42Z)
• Solving Inverse Problems With Deep Neural Networks -- Robustness Included? [3.867363075280544]
  Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks. In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts. This article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems.
  arXiv  Detail & Related papers  (2020-11-09T09:33:07Z)
• Deep Multi-Task Learning for Cooperative NOMA: System Design and Principles [52.79089414630366]
  We develop a novel deep cooperative NOMA scheme, drawing upon the recent advances in deep learning (DL) We develop a novel hybrid-cascaded deep neural network (DNN) architecture such that the entire system can be optimized in a holistic manner.
  arXiv  Detail & Related papers  (2020-07-27T12:38:37Z)
• Towards a practical measure of interference for reinforcement learning [37.1734757628306]
  Catastrophic interference is common in many network-based learning systems. We provide a definition of interference for control in reinforcement learning. Our new interference measure allows us to ask novel scientific questions about commonly used deep learning architectures.
  arXiv  Detail & Related papers  (2020-07-07T22:02:00Z)
• Interference and Generalization in Temporal Difference Learning [86.31598155056035]
  We study the link between generalization and interference in temporal-difference (TD) learning. We find that TD easily leads to low-interference, under-generalizing parameters, while the effect seems reversed in supervised learning.
  arXiv  Detail & Related papers  (2020-03-13T15:49:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.