Information-Bottleneck-Based Behavior Representation Learning for
Multi-agent Reinforcement learning
- URL: http://arxiv.org/abs/2109.14188v1
- Date: Wed, 29 Sep 2021 04:22:49 GMT
- Title: Information-Bottleneck-Based Behavior Representation Learning for
Multi-agent Reinforcement learning
- Authors: Yue Jin, Shuangqing Wei, Jian Yuan, Xudong Zhang
- Abstract summary: In deep reinforcement learning, extracting sufficient and compact information of other agents is critical to attain efficient convergence and scalability of an algorithm.
We present Information-Bottleneck-based Other agents' behavior Representation learning for Multi-agent reinforcement learning (IBORM) to explicitly seek low-dimensional mapping encoder.
- Score: 16.024781473545055
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In multi-agent deep reinforcement learning, extracting sufficient and compact
information of other agents is critical to attain efficient convergence and
scalability of an algorithm. In canonical frameworks, distilling of such
information is often done in an implicit and uninterpretable manner, or
explicitly with cost functions not able to reflect the relationship between
information compression and utility in representation. In this paper, we
present Information-Bottleneck-based Other agents' behavior Representation
learning for Multi-agent reinforcement learning (IBORM) to explicitly seek
low-dimensional mapping encoder through which a compact and informative
representation relevant to other agents' behaviors is established. IBORM
leverages the information bottleneck principle to compress observation
information, while retaining sufficient information relevant to other agents'
behaviors used for cooperation decision. Empirical results have demonstrated
that IBORM delivers the fastest convergence rate and the best performance of
the learned policies, as compared with implicit behavior representation
learning and explicit behavior representation learning without explicitly
considering information compression and utility.
Related papers
- Representation Learning with Conditional Information Flow Maximization [29.36409607847339]
This paper proposes an information-theoretic representation learning framework, named conditional information flow.
It promotes learned representations have good feature uniformity and sufficient predictive ability.
Experiments show that the learned representations are more sufficient, robust and transferable.
arXiv Detail & Related papers (2024-06-08T16:19:18Z) - Self-Supervised Representation Learning with Meta Comprehensive
Regularization [11.387994024747842]
We introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks.
We update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features.
We provide theoretical support for our proposed method from information theory and causal counterfactual perspective.
arXiv Detail & Related papers (2024-03-03T15:53:48Z) - Knowledge-Enhanced Hierarchical Information Correlation Learning for
Multi-Modal Rumor Detection [82.94413676131545]
We propose a novel knowledge-enhanced hierarchical information correlation learning approach (KhiCL) for multi-modal rumor detection.
KhiCL exploits cross-modal joint dictionary to transfer the heterogeneous unimodality features into the common feature space.
It extracts visual and textual entities from images and text, and designs a knowledge relevance reasoning strategy.
arXiv Detail & Related papers (2023-06-28T06:08:20Z) - Recognizable Information Bottleneck [31.993478081354958]
Information Bottlenecks (IBs) learn representations that generalize to unseen data by information compression.
IBs are practically unable to guarantee generalization in real-world scenarios due to the vacuous generalization bound.
We propose a Recognizable Information Bottleneck (RIB) which regularizes the recognizability of representations through a recognizability critic.
arXiv Detail & Related papers (2023-04-28T03:55:33Z) - Representation Learning in Deep RL via Discrete Information Bottleneck [39.375822469572434]
We study how information bottlenecks can be used to construct latent states efficiently in the presence of task-irrelevant information.
We propose architectures that utilize variational and discrete information bottlenecks, coined as RepDIB, to learn structured factorized representations.
arXiv Detail & Related papers (2022-12-28T14:38:12Z) - Variational Distillation for Multi-View Learning [104.17551354374821]
We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning.
Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
arXiv Detail & Related papers (2022-06-20T03:09:46Z) - An Empirical Investigation of Representation Learning for Imitation [76.48784376425911]
Recent work in vision, reinforcement learning, and NLP has shown that auxiliary representation learning objectives can reduce the need for large amounts of expensive, task-specific data.
We propose a modular framework for constructing representation learning algorithms, then use our framework to evaluate the utility of representation learning for imitation.
arXiv Detail & Related papers (2022-05-16T11:23:42Z) - Adaptive Discrete Communication Bottlenecks with Dynamic Vector
Quantization [76.68866368409216]
We propose learning to dynamically select discretization tightness conditioned on inputs.
We show that dynamically varying tightness in communication bottlenecks can improve model performance on visual reasoning and reinforcement learning tasks.
arXiv Detail & Related papers (2022-02-02T23:54:26Z) - Which Mutual-Information Representation Learning Objectives are
Sufficient for Control? [80.2534918595143]
Mutual information provides an appealing formalism for learning representations of data.
This paper formalizes the sufficiency of a state representation for learning and representing the optimal policy.
Surprisingly, we find that two of these objectives can yield insufficient representations given mild and common assumptions on the structure of the MDP.
arXiv Detail & Related papers (2021-06-14T10:12:34Z) - Robust Representation Learning via Perceptual Similarity Metrics [18.842322467828502]
Contrastive Input Morphing (CIM) is a representation learning framework that learns input-space transformations of the data.
We show that CIM is complementary to other mutual information-based representation learning techniques.
arXiv Detail & Related papers (2021-06-11T21:45:44Z) - A Theory of Usable Information Under Computational Constraints [103.5901638681034]
We propose a new framework for reasoning about information in complex systems.
Our foundation is based on a variational extension of Shannon's information theory.
We show that by incorporating computational constraints, $mathcalV$-information can be reliably estimated from data.
arXiv Detail & Related papers (2020-02-25T06:09:30Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.