Reinforcement Learning with Efficient Active Feature Acquisition
- URL: http://arxiv.org/abs/2011.00825v1
- Date: Mon, 2 Nov 2020 08:46:27 GMT
- Title: Reinforcement Learning with Efficient Active Feature Acquisition
- Authors: Haiyan Yin and Yingzhen Li and Sinno Jialin Pan and Cheng Zhang and
Sebastian Tschiatschek
- Abstract summary: In real-life, information acquisition might correspond to performing a medical test on a patient.
We propose a model-based reinforcement learning framework that learns an active feature acquisition policy.
Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states.
- Score: 59.91808801541007
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Solving real-life sequential decision making problems under partial
observability involves an exploration-exploitation problem. To be successful,
an agent needs to efficiently gather valuable information about the state of
the world for making rewarding decisions. However, in real-life, acquiring
valuable information is often highly costly, e.g., in the medical domain,
information acquisition might correspond to performing a medical test on a
patient. This poses a significant challenge for the agent to perform optimally
for the task while reducing the cost for information acquisition. In this
paper, we propose a model-based reinforcement learning framework that learns an
active feature acquisition policy to solve the exploration-exploitation problem
during its execution. Key to the success is a novel sequential variational
auto-encoder that learns high-quality representations from partially observed
states, which are then used by the policy to maximize the task reward in a cost
efficient manner. We demonstrate the efficacy of our proposed framework in a
control domain as well as using a medical simulator. In both tasks, our
proposed method outperforms conventional baselines and results in policies with
greater cost efficiency.
Related papers
- Amortized nonmyopic active search via deep imitation learning [16.037812098340343]
Active search formalizes a specialized active learning setting where the goal is to collect members of a rare, valuable class.
We study the amortization of this policy by training a neural network to learn to search.
Our network, trained on synthetic data, learns a beneficial search strategy that yields nonmyopic decisions.
arXiv Detail & Related papers (2024-05-23T20:10:29Z) - WESE: Weak Exploration to Strong Exploitation for LLM Agents [95.6720931773781]
This paper proposes a novel approach, Weak Exploration to Strong Exploitation (WESE) to enhance LLM agents in solving open-world interactive tasks.
WESE involves decoupling the exploration and exploitation process, employing a cost-effective weak agent to perform exploration tasks for global knowledge.
A knowledge graph-based strategy is then introduced to store the acquired knowledge and extract task-relevant knowledge, enhancing the stronger agent in success rate and efficiency for the exploitation task.
arXiv Detail & Related papers (2024-04-11T03:31:54Z) - Demonstration-Guided Reinforcement Learning with Efficient Exploration
for Task Automation of Surgical Robot [54.80144694888735]
We introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm.
Our method estimates expert-like behaviors with higher values to facilitate productive interactions.
Experiments on $10$ surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements.
arXiv Detail & Related papers (2023-02-20T05:38:54Z) - Learning from Guided Play: A Scheduled Hierarchical Approach for
Improving Exploration in Adversarial Imitation Learning [7.51557557629519]
We present Learning from Guided Play (LfGP), a framework in which we leverage expert demonstrations of, in addition to a main task, multiple auxiliary tasks.
This affords many benefits: learning efficiency is improved for main tasks with challenging bottleneck transitions, expert data becomes reusable between tasks, and transfer learning through the reuse of learned auxiliary task models becomes possible.
arXiv Detail & Related papers (2021-12-16T14:58:08Z) - MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven
Reinforcement Learning [65.52675802289775]
We show that an uncertainty aware classifier can solve challenging reinforcement learning problems.
We propose a novel method for computing the normalized maximum likelihood (NML) distribution.
We show that the resulting algorithm has a number of intriguing connections to both count-based exploration methods and prior algorithms for learning reward functions.
arXiv Detail & Related papers (2021-07-15T08:19:57Z) - Online reinforcement learning with sparse rewards through an active
inference capsule [62.997667081978825]
This paper introduces an active inference agent which minimizes the novel free energy of the expected future.
Our model is capable of solving sparse-reward problems with a very high sample efficiency.
We also introduce a novel method for approximating the prior model from the reward function, which simplifies the expression of complex objectives.
arXiv Detail & Related papers (2021-06-04T10:03:36Z) - Coverage as a Principle for Discovering Transferable Behavior in
Reinforcement Learning [16.12658895065585]
We argue that representation alone is not enough for efficient transfer in challenging domains and explore how to transfer knowledge through behavior.
The behavior of pre-trained policies may be used for solving the task at hand (exploitation) or for collecting useful data to solve the problem (exploration)
arXiv Detail & Related papers (2021-02-24T16:51:02Z) - Cost-effective Variational Active Entity Resolution [4.238343046459798]
We devise an entity resolution method that builds on the robustness conferred by deep autoencoders to reduce human-involvement costs.
Specifically, we reduce the cost of training deep entity resolution models by performing unsupervised representation learning.
Finally, we reduce the cost of labelling training data through an active learning approach that builds on the properties conferred by the use of deep autoencoders.
arXiv Detail & Related papers (2020-11-20T13:47:11Z) - Cost-Sensitive Portfolio Selection via Deep Reinforcement Learning [100.73223416589596]
We propose a cost-sensitive portfolio selection method with deep reinforcement learning.
Specifically, a novel two-stream portfolio policy network is devised to extract both price series patterns and asset correlations.
A new cost-sensitive reward function is developed to maximize the accumulated return and constrain both costs via reinforcement learning.
arXiv Detail & Related papers (2020-03-06T06:28:17Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.