Meta Off-Policy Estimation

• URL: http://arxiv.org/abs/2508.07914v1
• Date: Mon, 11 Aug 2025 12:31:13 GMT
• Title: Meta Off-Policy Estimation
• Authors: Olivier Jeunen,
• Abstract summary: Off-policy estimation (OPE) methods enable unbiased offline evaluation of recommender systems.<n>We take an alternative perspective to combine a set of OPE estimators and their associated confidence intervals into a single, more accurate estimate.<n>We validate our method on both simulated and real-world data, demonstrating improved statistical efficiency over existing individual estimators.
• Score: 3.988614978933934
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Off-policy estimation (OPE) methods enable unbiased offline evaluation of recommender systems, directly estimating the online reward some target policy would have obtained, from offline data and with statistical guarantees. The theoretical elegance of the framework combined with practical successes have led to a surge of interest, with many competing estimators now available to practitioners and researchers. Among these, Doubly Robust methods provide a prominent strategy to combine value- and policy-based estimators. In this work, we take an alternative perspective to combine a set of OPE estimators and their associated confidence intervals into a single, more accurate estimate. Our approach leverages a correlated fixed-effects meta-analysis framework, explicitly accounting for dependencies among estimators that arise due to shared data. This yields a best linear unbiased estimate (BLUE) of the target policy's value, along with an appropriately conservative confidence interval that reflects inter-estimator correlation. We validate our method on both simulated and real-world data, demonstrating improved statistical efficiency over existing individual estimators.

Related papers

• Off-Policy Evaluation and Learning for Matching Markets [15.585641615174623]
  Off-Policy Evaluation (OPE) plays a crucial role by enabling the evaluation of recommendation policies using only offline logged data.<n>We propose novel OPE estimators, textitDiPS and textitDPR, specifically designed for matching markets.<n>Our methods combine elements of the Direct Method (DM), Inverse Propensity Score (IPS), and Doubly Robust (DR) estimators.
  arXiv  Detail & Related papers  (2025-07-18T02:23:37Z)
• Generalized Linear Bandits: Almost Optimal Regret with One-Pass Update [60.414548453838506]
  We study the generalized linear bandit (GLB) problem, a contextual multi-armed bandit framework that extends the classical linear model by incorporating a non-linear link function.<n>GLBs are widely applicable to real-world scenarios, but their non-linear nature introduces significant challenges in achieving both computational and statistical efficiency.<n>We propose a jointly efficient algorithm that attains a nearly optimal regret bound with $mathcalO(1)$ time and space complexities per round.
  arXiv  Detail & Related papers  (2025-07-16T02:24:21Z)
• Distributionally Robust Policy Evaluation under General Covariate Shift in Contextual Bandits [31.571978291138866]
  We introduce a distributionally robust approach that enhances the reliability of offline policy evaluation in contextual bandits. Our method aims to deliver robust policy evaluation results in the presence of discrepancies in both context and policy distribution.
  arXiv  Detail & Related papers  (2024-01-21T00:42:06Z)
• Distributional Off-Policy Evaluation for Slate Recommendations [19.22972996548473]
  We propose an estimator for the complete off-policy performance distribution for slates. We validate the efficacy of our method empirically on synthetic data as well as on a slate recommendation simulator constructed from real-world data.
  arXiv  Detail & Related papers  (2023-08-27T17:58:32Z)
• Provable Offline Preference-Based Reinforcement Learning [95.00042541409901]
  We investigate the problem of offline Preference-based Reinforcement Learning (PbRL) with human feedback. We consider the general reward setting where the reward can be defined over the whole trajectory. We introduce a new single-policy concentrability coefficient, which can be upper bounded by the per-trajectory concentrability.
  arXiv  Detail & Related papers  (2023-05-24T07:11:26Z)
• Uncertainty-Aware Instance Reweighting for Off-Policy Learning [63.31923483172859]
  We propose a Uncertainty-aware Inverse Propensity Score estimator (UIPS) for improved off-policy learning. Experiment results on synthetic and three real-world recommendation datasets demonstrate the advantageous sample efficiency of the proposed UIPS estimator.
  arXiv  Detail & Related papers  (2023-03-11T11:42:26Z)
• Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation [53.83642844626703]
  We provide a unifying framework for estimating higher-order derivatives of value functions, based on off-policy evaluation. Our framework interprets a number of prior approaches as special cases and elucidates the bias and variance trade-off of Hessian estimates.
  arXiv  Detail & Related papers  (2021-06-24T15:58:01Z)
• Reliable Off-policy Evaluation for Reinforcement Learning [53.486680020852724]
  In a sequential decision-making problem, off-policy evaluation estimates the expected cumulative reward of a target policy. We propose a novel framework that provides robust and optimistic cumulative reward estimates using one or multiple logged data.
  arXiv  Detail & Related papers  (2020-11-08T23:16:19Z)
• CoinDICE: Off-Policy Confidence Interval Estimation [107.86876722777535]
  We study high-confidence behavior-agnostic off-policy evaluation in reinforcement learning. We show in a variety of benchmarks that the confidence interval estimates are tighter and more accurate than existing methods.
  arXiv  Detail & Related papers  (2020-10-22T12:39:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.