Quantifying Potential Observation Missingness in Inverse Reinforcement Learning

• URL: http://arxiv.org/abs/2605.12831v1
• Date: Tue, 12 May 2026 23:56:36 GMT
• Title: Quantifying Potential Observation Missingness in Inverse Reinforcement Learning
• Abstract summary: Inverse reinforcement learning (IRL) infers reward functions from demonstrations.<n>We show that recorded data may be missing observations that were available to the original decision-maker.<n>In use-inspired settings such as healthcare, this can make expert actions appear suboptimal.
• Score: 12.208013534038324
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inverse reinforcement learning (IRL), which infers reward functions from demonstrations, is a valuable tool for modeling and understanding decision-making behavior. Many variants of IRL have been developed to capture complexities of human decision-making, such as subjective beliefs, imperfect planning, and dynamic goals. However, an often-overlooked issue in real-world behavioral datasets is that the recorded data may be missing observations that were available to the original decision-maker. In use-inspired settings such as healthcare, this can make expert actions appear suboptimal, even when they were near-optimal given the information available at the time. As a result, the rewards learned by standard IRL may be misleading. In this paper, we identify the minimal perturbations to the recorded observations needed for the expert's actions to appear optimal. We develop a practical algorithm for this problem and demonstrate its utility for quantifying the possible extent of missing observations in behavioral datasets through extensive experiments on synthetic navigation tasks, a cancer treatment simulator, and ICU treatment data.

Related papers

• Can In-Context Learning Support Intrinsic Curiosity? [25.43664318918942]
  We investigate whether the emergent in-context learning capabilities of sequence models can eliminate this bottleneck.<n>Specifically, we evaluate whether an exploration policy can be trained to maximize learning progress.<n>We prove a positive result for a broad subclass of non-temporal settings.
  arXiv  Detail & Related papers  (2026-06-17T18:11:06Z)
• Summary of the Unusual Activity Recognition Challenge for Developmental Disability Support [1.369513462160388]
  The challenge aims to address the critical need for automated recognition of unusual behaviors in facilities for individuals with developmental disabilities.<n>Teams were tasked with distinguishing between normal and unusual activities based on skeleton keypoints extracted from video recordings of simulated scenarios.<n>The dataset reflects real-world imbalance and temporal irregularities in behavior, and the evaluation adopted a Leave-One-Subject-Out (LOSO) strategy to ensure subject-agnostic generalization.
  arXiv  Detail & Related papers  (2026-01-21T04:41:35Z)
• From Physics to Machine Learning and Back: Part II - Learning and Observational Bias in PHM [52.64097278841485]
  Review examines how incorporating learning and observational biases through physics-informed modeling and data strategies can guide models toward physically consistent and reliable predictions.<n>Fast adaptation methods including meta-learning and few-shot learning are reviewed alongside domain generalization techniques.
  arXiv  Detail & Related papers  (2025-09-25T14:15:43Z)
• Stochastic Encodings for Active Feature Acquisition [100.47043816019888]
  Active Feature Acquisition is an instance-wise, sequential decision making problem.<n>The aim is to dynamically select which feature to measure based on current observations, independently for each test instance.<n>Common approaches either use Reinforcement Learning, which experiences training difficulties, or greedily maximize the conditional mutual information of the label and unobserved features, which makes myopic.<n>We introduce a latent variable model, trained in a supervised manner. Acquisitions are made by reasoning about the features across many possible unobserved realizations in a latent space.
  arXiv  Detail & Related papers  (2025-08-03T23:48:46Z)
• Value from Observations: Towards Large-Scale Imitation Learning via Self-Improvement [19.883973457999282]
  Imitation Learning from Observation (IfO) offers a powerful way to learn behaviors at large-scale.<n>This paper investigates idealized scenarios with mostly bimodal-quality data distributions and introduces a method to learn from such data.<n>Our method adapts RL-based imitation learning to action-free demonstrations, using a value function to transfer information between expert and non-expert data.
  arXiv  Detail & Related papers  (2025-07-09T09:55:23Z)
• Offline Action-Free Learning of Ex-BMDPs by Comparing Diverse Datasets [87.62730694973696]
  This paper introduces CRAFT, a sample-efficient algorithm leveraging differences in controllable feature dynamics across agents to learn representations.<n>We provide theoretical guarantees for CRAFT's performance and demonstrate its feasibility on a toy example.
  arXiv  Detail & Related papers  (2025-03-26T22:05:57Z)
• Green LIME: Improving AI Explainability through Design of Experiments [44.99833362998488]
  Local Interpretable Model-agnostic Explanations (LIME) provides explanations by generating new data points near the instance of interest and passing them through the model.<n>LIME is highly versatile and can be applied to a wide range of models and datasets.
  arXiv  Detail & Related papers  (2025-02-18T11:15:04Z)
• Explanatory Model Monitoring to Understand the Effects of Feature Shifts on Performance [61.06245197347139]
  We propose a novel approach to explain the behavior of a black-box model under feature shifts. We refer to our method that combines concepts from Optimal Transport and Shapley Values as Explanatory Performance Estimation.
  arXiv  Detail & Related papers  (2024-08-24T18:28:19Z)
• Reinforcement Learning from Passive Data via Latent Intentions [86.4969514480008]
  We show that passive data can still be used to learn features that accelerate downstream RL. Our approach learns from passive data by modeling intentions. Our experiments demonstrate the ability to learn from many forms of passive data, including cross-embodiment video data and YouTube videos.
  arXiv  Detail & Related papers  (2023-04-10T17:59:05Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.