Active hypothesis testing in unknown environments using recurrent neural networks and model free reinforcement learning

• URL: http://arxiv.org/abs/2303.10623v2
• Date: Tue, 6 Jun 2023 11:29:51 GMT
• Title: Active hypothesis testing in unknown environments using recurrent neural networks and model free reinforcement learning
• Authors: George Stamatelis, Nicholas Kalouptsidis
• Abstract summary: We make no assumptions about the prior probability, the action and observation sets, and the observation generating process. Our method can be used in any environment even if it has continuous observations or actions, and performs competitively and sometimes better than the Chernoff test.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A combination of deep reinforcement learning and supervised learning is proposed for the problem of active sequential hypothesis testing in completely unknown environments. We make no assumptions about the prior probability, the action and observation sets, and the observation generating process. Our method can be used in any environment even if it has continuous observations or actions, and performs competitively and sometimes better than the Chernoff test, in both finite and infinite horizon problems, despite not having access to the environment dynamics.

Related papers

• In-Context Learning for Pure Exploration in Continuous Spaces [26.001092687873125]
  In active sequential testing, also termed pure exploration, a learner is tasked with the goal to adaptively acquire information.<n>We introduce C-ICPE-TS, an algorithm that meta-trains deep neural policies to map observation histories to the next continuous query action.<n>At inference time, C-ICPE-TS actively gathers evidence on previously unseen tasks and infers the true hypothesis without parameter updates or explicit hand-crafted information models.
  arXiv  Detail & Related papers  (2026-02-20T04:20:47Z)
• Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling [53.7117640028211]
  We present a geometry-constrained probabilistic modeling treatment to resolve the identified issues. We incorporate a suite of critical geometric properties to impose proper constraints on the layout of constructed embedding space. A spectral graph-theoretic method is devised to estimate the number of potential novel classes.
  arXiv  Detail & Related papers  (2024-03-02T00:56:05Z)
• Natural Mitigation of Catastrophic Interference: Continual Learning in Power-Law Learning Environments [2.714641498775159]
  We show that in naturalistic environments, the probability of encountering a task decreases as a power-law of the time since it was last performed. We evaluate the extent of the natural mitigation of CI when training models in power-law environments, similar to those humans face.
  arXiv  Detail & Related papers  (2024-01-18T22:06:38Z)
• Conditional Kernel Imitation Learning for Continuous State Environments [9.750698192309978]
  We introduce a novel conditional kernel density estimation-based imitation learning framework. We show consistently superior empirical performance over many state-of-the-art IL algorithms.
  arXiv  Detail & Related papers  (2023-08-24T05:26:42Z)
• A Double Machine Learning Approach to Combining Experimental and Observational Data [59.29868677652324]
  We propose a double machine learning approach to combine experimental and observational studies. Our framework tests for violations of external validity and ignorability under milder assumptions.
  arXiv  Detail & Related papers  (2023-07-04T02:53:11Z)
• Active Exploration via Experiment Design in Markov Chains [86.41407938210193]
  A key challenge in science and engineering is to design experiments to learn about some unknown quantity of interest. We propose an algorithm that efficiently selects policies whose measurement allocation converges to the optimal one. In addition to our theoretical analysis, we showcase our framework on applications in ecological surveillance and pharmacology.
  arXiv  Detail & Related papers  (2022-06-29T00:04:40Z)
• Disentangling Epistemic and Aleatoric Uncertainty in Reinforcement Learning [35.791555387656956]
  Aleatoric uncertainty results from the irreducible environment leading to inherently risky states and actions. Epistemic uncertainty results from the limited information accumulated during learning to make informed decisions. Characterizing aleatoric and uncertainty can be used to speed up learning in a training environment, improve generalization to similar testing environments, and flag unfamiliar behavior in anomalous testing environments.
  arXiv  Detail & Related papers  (2022-06-03T13:20:16Z)
• Continual Predictive Learning from Videos [100.27176974654559]
  We study a new continual learning problem in the context of video prediction. We propose the continual predictive learning (CPL) approach, which learns a mixture world model via predictive experience replay. We construct two new benchmarks based on RoboNet and KTH, in which different tasks correspond to different physical robotic environments or human actions.
  arXiv  Detail & Related papers  (2022-04-12T08:32:26Z)
• Provable RL with Exogenous Distractors via Multistep Inverse Dynamics [85.52408288789164]
  Real-world applications of reinforcement learning (RL) require the agent to deal with high-dimensional observations such as those generated from a megapixel camera. Prior work has addressed such problems with representation learning, through which the agent can provably extract endogenous, latent state information from raw observations. However, such approaches can fail in the presence of temporally correlated noise in the observations.
  arXiv  Detail & Related papers  (2021-10-17T15:21:27Z)
• Generalized Chernoff Sampling for Active Testing, Active Regression and Structured Bandit Algorithms [16.19565714525819]
  This paper studies active learning and best-arm identification in structured bandit settings. We obtain a novel sample bound complexity for Chernoff's original active testing procedure.
  arXiv  Detail & Related papers  (2020-12-15T03:44:18Z)
• Attribute-Guided Adversarial Training for Robustness to Natural Perturbations [64.35805267250682]
  We propose an adversarial training approach which learns to generate new samples so as to maximize exposure of the classifier to the attributes-space. Our approach enables deep neural networks to be robust against a wide range of naturally occurring perturbations.
  arXiv  Detail & Related papers  (2020-12-03T10:17:30Z)
• A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning [8.188575923130662]
  We argue that notable lessons from open set recognition, the identification of statistically deviating data outside of the observed dataset, and the adjacent field of active learning, are frequently overlooked in the deep learning era. Our results show that this not only benefits each individual paradigm, but highlights the natural synergies in a common framework.
  arXiv  Detail & Related papers  (2020-09-03T16:56:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.