Observation-Augmented Contextual Multi-Armed Bandits for Robotic Exploration with Uncertain Semantic Data

• URL: http://arxiv.org/abs/2312.12583v1
• Date: Tue, 19 Dec 2023 20:28:42 GMT
• Title: Observation-Augmented Contextual Multi-Armed Bandits for Robotic Exploration with Uncertain Semantic Data
• Authors: Shohei Wakayama and Nisar Ahmed
• Abstract summary: We introduce a new variant of contextual multi-armed bandits called observation-augmented CMABs (OA-CMABs) OA-CMABs model the expected option outcomes as a function of context features and hidden parameters. We propose a robust Bayesian inference process for OA-CMABs that is based on the concept of probabilistic data validation.
• Score: 7.795929277007235
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: For robotic decision-making under uncertainty, the balance between exploitation and exploration of available options must be carefully taken into account. In this study, we introduce a new variant of contextual multi-armed bandits called observation-augmented CMABs (OA-CMABs) wherein a decision-making agent can utilize extra outcome observations from an external information source. CMABs model the expected option outcomes as a function of context features and hidden parameters, which are inferred from previous option outcomes. In OA-CMABs, external observations are also a function of context features and thus provide additional evidence about the hidden parameters. Yet, if an external information source is error-prone, the resulting posterior updates can harm decision-making performance unless the presence of errors is considered. To this end, we propose a robust Bayesian inference process for OA-CMABs that is based on the concept of probabilistic data validation. Our approach handles complex mixture model parameter priors and hybrid observation likelihoods for semantic data sources, allowing us to develop validation algorithms based on recently develop probabilistic semantic data association techniques. Furthermore, to more effectively cope with the combined sources of uncertainty in OA-CMABs, we derive a new active inference algorithm for option selection based on expected free energy minimization. This generalizes previous work on active inference for bandit-based robotic decision-making by accounting for faulty observations and non-Gaussian inference. Our approaches are demonstrated on a simulated asynchronous search site selection problem for space exploration. The results show that even if incorrect observations are provided by external information sources, efficient decision-making and robust parameter inference are still achieved in a wide variety of experimental conditions.

Related papers

• Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators [19.053826145863113]
  This paper introduces a Distributionally Robust Metric (DRM) for CATE estimator selection. DRM is nuisance-free, eliminating the need to fit models for nuisance parameters. It effectively prioritizes the selection of a distributionally robust CATE estimator.
  arXiv  Detail & Related papers  (2024-02-28T15:12:24Z)
• Calibrating Neural Simulation-Based Inference with Differentiable Coverage Probability [50.44439018155837]
  We propose to include a calibration term directly into the training objective of the neural model. By introducing a relaxation of the classical formulation of calibration error we enable end-to-end backpropagation. It is directly applicable to existing computational pipelines allowing reliable black-box posterior inference.
  arXiv  Detail & Related papers  (2023-10-20T10:20:45Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• Approximating Counterfactual Bounds while Fusing Observational, Biased and Randomised Data Sources [64.96984404868411]
  We address the problem of integrating data from multiple, possibly biased, observational and interventional studies. We show that the likelihood of the available data has no local maxima. We then show how the same approach can address the general case of multiple datasets.
  arXiv  Detail & Related papers  (2023-07-31T11:28:24Z)
• Learning to Bound Counterfactual Inference in Structural Causal Models from Observational and Randomised Data [64.96984404868411]
  We derive a likelihood characterisation for the overall data that leads us to extend a previous EM-based algorithm. The new algorithm learns to approximate the (unidentifiability) region of model parameters from such mixed data sources. It delivers interval approximations to counterfactual results, which collapse to points in the identifiable case.
  arXiv  Detail & Related papers  (2022-12-06T12:42:11Z)
• Rethinking Missing Data: Aleatoric Uncertainty-Aware Recommendation [59.500347564280204]
  We propose a new Aleatoric Uncertainty-aware Recommendation (AUR) framework. AUR consists of a new uncertainty estimator along with a normal recommender model. As the chance of mislabeling reflects the potential of a pair, AUR makes recommendations according to the uncertainty.
  arXiv  Detail & Related papers  (2022-09-22T04:32:51Z)
• Active Inference for Autonomous Decision-Making with Contextual Multi-Armed Bandits [1.3670071336891754]
  In autonomous robotic decision-making under uncertainty, the tradeoff between exploitation and exploration of available options must be considered. In this study, we apply active inference, which has been actively studied in the field of neuroscience in recent years, as an alternative action selection strategy for CMABs.
  arXiv  Detail & Related papers  (2022-09-19T17:11:21Z)
• Error-based Knockoffs Inference for Controlled Feature Selection [49.99321384855201]
  We propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees.
  arXiv  Detail & Related papers  (2022-03-09T01:55:59Z)
• Federated Estimation of Causal Effects from Observational Data [19.657789891394504]
  We present a novel framework for causal inference with federated data sources. We assess and integrate local causal effects from different private data sources without centralizing them.
  arXiv  Detail & Related papers  (2021-05-31T08:06:00Z)
• On Bayesian Search for the Feasible Space Under Computationally Expensive Constraints [0.0]
  We propose a novel acquisition function that combines the probability that a solution lies at the boundary between feasible and infeasible spaces. Experiments confirmed the efficacy of the proposed function.
  arXiv  Detail & Related papers  (2020-04-23T10:22:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.