Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition

• URL: http://arxiv.org/abs/2302.13960v1
• Date: Mon, 27 Feb 2023 17:02:11 GMT
• Title: Acquisition Conditioned Oracle for Nongreedy Active Feature Acquisition
• Authors: Michael Valancius, Max Lennon, Junier Oliva
• Abstract summary: We develop methodology for active feature acquisition (AFA) We show that we can bypass many challenges with a novel, nonparametric oracle based approach.
• Score: 16.350351668269415
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We develop novel methodology for active feature acquisition (AFA), the study of how to sequentially acquire a dynamic (on a per instance basis) subset of features that minimizes acquisition costs whilst still yielding accurate predictions. The AFA framework can be useful in a myriad of domains, including health care applications where the cost of acquiring additional features for a patient (in terms of time, money, risk, etc.) can be weighed against the expected improvement to diagnostic performance. Previous approaches for AFA have employed either: deep learning RL techniques, which have difficulty training policies in the AFA MDP due to sparse rewards and a complicated action space; deep learning surrogate generative models, which require modeling complicated multidimensional conditional distributions; or greedy policies, which fail to account for how joint feature acquisitions can be informative together for better predictions. In this work we show that we can bypass many of these challenges with a novel, nonparametric oracle based approach, which we coin the acquisition conditioned oracle (ACO). Extensive experiments show the superiority of the ACO to state-of-the-art AFA methods when acquiring features for both predictions and general decision-making.

Related papers

• Learning Off-policy with Model-based Intrinsic Motivation For Active Online Exploration [15.463313629574111]
  This paper investigates how to achieve sample-efficient exploration in continuous control tasks. We introduce an RL algorithm that incorporates a predictive model and off-policy learning elements. We derive an intrinsic reward without incurring parameters overhead.
  arXiv  Detail & Related papers  (2024-03-31T11:39:11Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Evaluation of Active Feature Acquisition Methods for Time-varying Feature Settings [6.082810456767599]
  Machine learning methods often assume that input features are available at no cost. In domains like healthcare, where acquiring features could be expensive harmful, it is necessary to balance a features acquisition against its predictive positivity. We present a problem of active feature acquisition performance evaluation (AFAPE)
  arXiv  Detail & Related papers  (2023-12-03T23:08:29Z)
• Improving Generalization of Alignment with Human Preferences through Group Invariant Learning [56.19242260613749]
  Reinforcement Learning from Human Feedback (RLHF) enables the generation of responses more aligned with human preferences. Previous work shows that Reinforcement Learning (RL) often exploits shortcuts to attain high rewards and overlooks challenging samples. We propose a novel approach that can learn a consistent policy via RL across various data groups or domains.
  arXiv  Detail & Related papers  (2023-10-18T13:54:15Z)
• Secrets of RLHF in Large Language Models Part I: PPO [81.01936993929127]
  Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. reinforcement learning with human feedback (RLHF) emerges as the pivotal technological paradigm underpinning this pursuit. In this report, we dissect the framework of RLHF, re-evaluate the inner workings of PPO, and explore how the parts comprising PPO algorithms impact policy agent training.
  arXiv  Detail & Related papers  (2023-07-11T01:55:24Z)
• Discovering Hierarchical Achievements in Reinforcement Learning via Contrastive Learning [17.28280896937486]
  We introduce a novel contrastive learning method, called achievement distillation, which strengthens the agent's ability to predict the next achievement. Our method exhibits a strong capacity for discovering hierarchical achievements and shows state-of-the-art performance on the challenging Crafter environment.
  arXiv  Detail & Related papers  (2023-07-07T09:47:15Z)
• Improved Regret for Efficient Online Reinforcement Learning with Linear Function Approximation [69.0695698566235]
  We study reinforcement learning with linear function approximation and adversarially changing cost functions. We present a computationally efficient policy optimization algorithm for the challenging general setting of unknown dynamics and bandit feedback.
  arXiv  Detail & Related papers  (2023-01-30T17:26:39Z)
• Active Surrogate Estimators: An Active Learning Approach to Label-Efficient Model Evaluation [59.7305309038676]
  We propose Active Surrogate Estimators (ASEs) for model evaluation. We find that ASEs offer greater label-efficiency than the current state-of-the-art.
  arXiv  Detail & Related papers  (2022-02-14T17:15:18Z)
• Towards Robust Active Feature Acquisition [14.785570635390744]
  Active feature acquisition (AFA) models deal with a small set of candidate features and have difficulty scaling to a large feature space. We propose several techniques to advance the current AFA approaches. Our framework can easily handle a large number of features using a hierarchical acquisition policy and is more robust to OOD inputs with the help of an OOD detector for partially observed data.
  arXiv  Detail & Related papers  (2021-07-09T01:06:13Z)
• Reinforcement Learning with Efficient Active Feature Acquisition [59.91808801541007]
  In real-life, information acquisition might correspond to performing a medical test on a patient. We propose a model-based reinforcement learning framework that learns an active feature acquisition policy. Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states.
  arXiv  Detail & Related papers  (2020-11-02T08:46:27Z)
• Active Feature Acquisition with Generative Surrogate Models [11.655069211977464]
  In this work, we consider models that perform active feature acquisition (AFA) and query the environment for unobserved features. Our work reformulates the Markov decision process (MDP) that underlies the AFA problem as a generative modeling task. We propose learning a generative surrogate model ( GSM) that captures the dependencies among input features to assess potential information gain from acquisitions.
  arXiv  Detail & Related papers  (2020-10-06T02:10:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.