A Bit Better? Quantifying Information for Bandit Learning
- URL: http://arxiv.org/abs/2102.09488v1
- Date: Thu, 18 Feb 2021 17:16:04 GMT
- Title: A Bit Better? Quantifying Information for Bandit Learning
- Authors: Adithya M. Devraj, Benjamin Van Roy, Kuang Xu
- Abstract summary: The information ratio offers an approach to assessing the efficacy with which an agent balances between exploration and exploitation.
Recent work has inspired consideration of alternative information measures, particularly for use in analysis of bandit learning algorithms to arrive at tighter regret bounds.
We investigate whether quantification of information via such alternatives can improve the realized performance of information-directed sampling.
- Score: 24.943571034827297
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: The information ratio offers an approach to assessing the efficacy with which
an agent balances between exploration and exploitation. Originally, this was
defined to be the ratio between squared expected regret and the mutual
information between the environment and action-observation pair, which
represents a measure of information gain. Recent work has inspired
consideration of alternative information measures, particularly for use in
analysis of bandit learning algorithms to arrive at tighter regret bounds. We
investigate whether quantification of information via such alternatives can
improve the realized performance of information-directed sampling, which aims
to minimize the information ratio.
Related papers
- On Bits and Bandits: Quantifying the Regret-Information Trade-off [62.64904903955711]
In interactive decision-making tasks, information can be acquired by direct interactions, through receiving indirect feedback, and from external knowledgeable sources.
We show that information from external sources, measured in bits, can be traded off for regret, measured in reward.
We introduce the first Bayesian regret lower bounds that depend on the information an agent accumulates.
arXiv Detail & Related papers (2024-05-26T14:18:38Z) - Collaborative Knowledge Infusion for Low-resource Stance Detection [83.88515573352795]
Target-related knowledge is often needed to assist stance detection models.
We propose a collaborative knowledge infusion approach for low-resource stance detection tasks.
arXiv Detail & Related papers (2024-03-28T08:32:14Z) - dugMatting: Decomposed-Uncertainty-Guided Matting [83.71273621169404]
We propose a decomposed-uncertainty-guided matting algorithm, which explores the explicitly decomposed uncertainties to efficiently and effectively improve the results.
The proposed matting framework relieves the requirement for users to determine the interaction areas by using simple and efficient labeling.
arXiv Detail & Related papers (2023-06-02T11:19:50Z) - Scalable Infomin Learning [39.77171117174905]
infomin learning aims to learn a representation with high utility while being uninformative about a specified target.
Recent works on infomin learning mainly use adversarial training, which involves training a neural network to estimate mutual information.
We propose a new infomin learning approach, which uses a novel proxy metric to mutual information.
arXiv Detail & Related papers (2023-02-21T14:40:25Z) - STEERING: Stein Information Directed Exploration for Model-Based
Reinforcement Learning [111.75423966239092]
We propose an exploration incentive in terms of the integral probability metric (IPM) between a current estimate of the transition model and the unknown optimal.
Based on KSD, we develop a novel algorithm algo: textbfSTEin information dirtextbfEcted exploration for model-based textbfReinforcement LearntextbfING.
arXiv Detail & Related papers (2023-01-28T00:49:28Z) - An Information Minimization Based Contrastive Learning Model for
Unsupervised Sentence Embeddings Learning [19.270283247740664]
We present an information minimization based contrastive learning (InforMin-CL) model for unsupervised sentence representation learning.
We find that information minimization can be achieved by simple contrast and reconstruction objectives.
arXiv Detail & Related papers (2022-09-22T12:07:35Z) - Information-Bottleneck-Based Behavior Representation Learning for
Multi-agent Reinforcement learning [16.024781473545055]
In deep reinforcement learning, extracting sufficient and compact information of other agents is critical to attain efficient convergence and scalability of an algorithm.
We present Information-Bottleneck-based Other agents' behavior Representation learning for Multi-agent reinforcement learning (IBORM) to explicitly seek low-dimensional mapping encoder.
arXiv Detail & Related papers (2021-09-29T04:22:49Z) - A Bayesian Framework for Information-Theoretic Probing [51.98576673620385]
We argue that probing should be seen as approximating a mutual information.
This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences.
This paper proposes a new framework to measure what we term Bayesian mutual information.
arXiv Detail & Related papers (2021-09-08T18:08:36Z) - Learning Bias-Invariant Representation by Cross-Sample Mutual
Information Minimization [77.8735802150511]
We propose a cross-sample adversarial debiasing (CSAD) method to remove the bias information misused by the target task.
The correlation measurement plays a critical role in adversarial debiasing and is conducted by a cross-sample neural mutual information estimator.
We conduct thorough experiments on publicly available datasets to validate the advantages of the proposed method over state-of-the-art approaches.
arXiv Detail & Related papers (2021-08-11T21:17:02Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.