Measuring and Mitigating Interference in Reinforcement Learning
- URL: http://arxiv.org/abs/2307.04887v1
- Date: Mon, 10 Jul 2023 20:20:20 GMT
- Title: Measuring and Mitigating Interference in Reinforcement Learning
- Authors: Vincent Liu, Han Wang, Ruo Yu Tao, Khurram Javed, Adam White, Martha
White
- Abstract summary: Catastrophic interference is common in many network-based learning systems.
We provide a definition and novel measure of interference for value-based reinforcement learning methods.
- Score: 30.38857177546063
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Catastrophic interference is common in many network-based learning systems,
and many proposals exist for mitigating it. Before overcoming interference we
must understand it better. In this work, we provide a definition and novel
measure of interference for value-based reinforcement learning methods such as
Fitted Q-Iteration and DQN. We systematically evaluate our measure of
interference, showing that it correlates with instability in control
performance, across a variety of network architectures. Our new interference
measure allows us to ask novel scientific questions about commonly used deep
learning architectures and study learning algorithms which mitigate
interference. Lastly, we outline a class of algorithms which we call
online-aware that are designed to mitigate interference, and show they do
reduce interference according to our measure and that they improve stability
and performance in several classic control environments.
Related papers
- A Two-Part Machine Learning Approach to Characterizing Network Interference in A/B Testing [4.000213034401085]
We introduce "causal network motifs" and utilize transparent machine learning models to characterize network interference patterns.
Our approach provides a comprehensive and automated solution to address network interference for A/B testing practitioners.
arXiv Detail & Related papers (2023-08-18T19:37:55Z) - Edge Intelligence Over the Air: Two Faces of Interference in Federated
Learning [95.31679010587473]
Federated edge learning is envisioned as the bedrock of enabling intelligence in next-generation wireless networks.
This article provides a comprehensive overview of the positive and negative effects of interference on over-the-air-based edge learning systems.
arXiv Detail & Related papers (2023-06-17T09:04:48Z) - Deep Reinforcement Learning for Interference Management in UAV-based 3D
Networks: Potentials and Challenges [137.47736805685457]
We show that interference can still be effectively mitigated even without knowing its channel information.
By harnessing interference, the proposed solutions enable the continued growth of civilian UAVs.
arXiv Detail & Related papers (2023-05-11T18:06:46Z) - Detecting Irregular Network Activity with Adversarial Learning and
Expert Feedback [14.188603782159372]
CAAD employs contrastive learning in an adversarial setup to learn effective representations of normal and anomalous behavior in wireless networks.
We conduct rigorous performance comparisons of CAAD with several state-of-the-art anomaly detection techniques and verify that CAAD yields a mean performance improvement of 92.84%.
arXiv Detail & Related papers (2022-10-01T20:44:14Z) - Large-Scale Sequential Learning for Recommender and Engineering Systems [91.3755431537592]
In this thesis, we focus on the design of an automatic algorithms that provide personalized ranking by adapting to the current conditions.
For the former, we propose novel algorithm called SAROS that take into account both kinds of feedback for learning over the sequence of interactions.
The proposed idea of taking into account the neighbour lines shows statistically significant results in comparison with the initial approach for faults detection in power grid.
arXiv Detail & Related papers (2022-05-13T21:09:41Z) - Interference Suppression Using Deep Learning: Current Approaches and
Open Challenges [2.179313476241343]
In this paper, we review a wide range of techniques that have used deep learning to suppress interference.
We provide comparison and guidelines for many different types of deep learning techniques in interference suppression.
In addition, we highlight challenges and potential future research directions for the successful adoption of deep learning in interference suppression.
arXiv Detail & Related papers (2021-12-16T16:07:42Z) - Solving Inverse Problems With Deep Neural Networks -- Robustness
Included? [3.867363075280544]
Recent works have pointed out instabilities of deep neural networks for several image reconstruction tasks.
In analogy to adversarial attacks in classification, it was shown that slight distortions in the input domain may cause severe artifacts.
This article sheds new light on this concern, by conducting an extensive study of the robustness of deep-learning-based algorithms for solving underdetermined inverse problems.
arXiv Detail & Related papers (2020-11-09T09:33:07Z) - Deep Multi-Task Learning for Cooperative NOMA: System Design and
Principles [52.79089414630366]
We develop a novel deep cooperative NOMA scheme, drawing upon the recent advances in deep learning (DL)
We develop a novel hybrid-cascaded deep neural network (DNN) architecture such that the entire system can be optimized in a holistic manner.
arXiv Detail & Related papers (2020-07-27T12:38:37Z) - Towards a practical measure of interference for reinforcement learning [37.1734757628306]
Catastrophic interference is common in many network-based learning systems.
We provide a definition of interference for control in reinforcement learning.
Our new interference measure allows us to ask novel scientific questions about commonly used deep learning architectures.
arXiv Detail & Related papers (2020-07-07T22:02:00Z) - Interference and Generalization in Temporal Difference Learning [86.31598155056035]
We study the link between generalization and interference in temporal-difference (TD) learning.
We find that TD easily leads to low-interference, under-generalizing parameters, while the effect seems reversed in supervised learning.
arXiv Detail & Related papers (2020-03-13T15:49:58Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.