Off-policy evaluation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 102 papers with code (265 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation
  17 Aug 2020 4 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  Our dataset is unique in that it contains a set of multiple logged bandit datasets collected by running different policies on the same platform.
• Off-Policy Evaluation for Large Action Spaces via Embeddings
  13 Feb 2022 3 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  Unfortunately, when the number of actions is large, existing OPE estimators -- most of which are based on inverse propensity score weighting -- degrade severely and can suffer from extreme bias and variance.
• Benchmarks for Deep Off-Policy Evaluation
  30 Mar 2021 3 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Off-policy evaluation (OPE) holds the promise of being able to leverage large, offline datasets for both evaluating and selecting complex policies for decision making.
• Optimal and Adaptive Off-policy Evaluation in Contextual Bandits
  4 Dec 2016 3 repositories listed
  We study the off-policy evaluation problem---estimating the value of a target policy using data collected by another policy---under the contextual bandit model.
• Balanced Off-Policy Evaluation for Personalized Pricing
  24 Feb 2023 2 repositories listed
  We consider a personalized pricing problem in which we have data consisting of feature information, historical pricing decisions, and binary realized demand.
• Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model
  3 Feb 2022 2 repositories listed
  We show that the proposed estimator is unbiased in more cases compared to existing estimators that make stronger assumptions.
• Evaluating the Robustness of Off-Policy Evaluation
  31 Aug 2021 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  Unfortunately, identifying a reliable estimator from results reported in research papers is often difficult because the current experimental procedure evaluates and compares the estimators' performance on a narrow set…
• Robust Generalization despite Distribution Shift via Minimum Discriminating Information
  8 Jun 2021 2 repositories listed
  Training models that perform well under distribution shifts is a central challenge in machine learning.
• Confident Off-Policy Evaluation and Selection through Self-Normalized Importance Weighting
  18 Jun 2020 2 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  We consider off-policy evaluation in the contextual bandit setting for the purpose of obtaining a robust off-policy selection strategy, where the selection strategy is evaluated based on the value of the chosen policy…
• Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning
  9 Jun 2019 2 repositories listed
  We propose new estimators for OPE based on empirical likelihood that are always more efficient than IS, SNIS, and DR and satisfy the same stability and boundedness properties as SNIS.
• DOLCE: Decomposing Off-Policy Evaluation/Learning into Lagged and Current Effects
  2 May 2025 1 repository listed
  Off-policy evaluation (OPE) and off-policy learning (OPL) for contextual bandit policies leverage historical data to evaluate and optimize a target policy.
• Trajectory World Models for Heterogeneous Environments
  3 Feb 2025 1 repository listed
  Heterogeneity in sensors and actuators across environments poses a significant challenge to building large-scale pre-trained world models on top of this low-dimensional sensor information.
• Two-way Deconfounder for Off-policy Evaluation in Causal Reinforcement Learning
  8 Dec 2024 1 repository listed Syntology ran 0 of 4 samples · 4 unverified · 4 pointer-only (licence)
  This paper studies off-policy evaluation (OPE) in the presence of unmeasured confounders.
• Minimum Empirical Divergence for Sub-Gaussian Linear Bandits
  31 Oct 2024 1 repository listed
  We propose a novel linear bandit algorithm called LinMED (Linear Minimum Empirical Divergence), which is a linear extension of the MED algorithm that was originally designed for multi-armed bandits.
• Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluation
  3 Oct 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Evaluating policies using off-policy data is crucial for applying reinforcement learning to real-world problems such as healthcare and autonomous driving.
• Causal Deepsets for Off-policy Evaluation under Spatial or Spatio-temporal Interferences
  25 Jul 2024 1 repository listed
  Off-policy evaluation (OPE) is widely applied in sectors such as pharmaceuticals and e-commerce to evaluate the efficacy of novel products or policies from offline datasets.
• Off-policy Evaluation with Deeply-abstracted States
  27 Jun 2024 1 repository listed
  Off-policy evaluation (OPE) is crucial for assessing a target policy's impact offline before its deployment.
• Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies
  29 May 2024 1 repository listed Syntology ran 5 of 8 samples · 3 unverified · 1 pointer-only (licence)
  To address this limitation, we propose to relax the deterministic target policy using a kernel and learn the kernel metrics that minimize the overall mean squared error of the estimated temporal difference update vector…
• Cross-Validated Off-Policy Evaluation
  24 May 2024 1 repository listed
  We study estimator selection and hyper-parameter tuning in off-policy evaluation.
• Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning
  23 May 2024 1 repository listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  This work investigates the offline formulation of the contextual bandit problem, where the goal is to leverage past interactions collected under a behavior policy to evaluate, select, and learn new, potentially…
• Long-term Off-Policy Evaluation and Learning
  24 Apr 2024 1 repository listed
  Existing approaches to this problem either need a restrictive assumption about the short-term outcomes called surrogacy or cannot effectively use short-term outcomes, which is inefficient.
• Predictive Performance Comparison of Decision Policies Under Confounding
  1 Apr 2024 1 repository listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
  However, it is challenging to compare predictive performance against an existing decision-making policy that is generally under-specified and dependent on unobservable factors.
• Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes
  29 Mar 2024 1 repository listed Syntology ran 0 of 3 samples · 3 unverified · 3 pointer-only (licence)
  We study the evaluation of a policy under best- and worst-case perturbations to a Markov decision process (MDP), using transition observations from the original MDP, whether they are generated under the same or a…
• Off-Policy Evaluation of Slate Bandit Policies via Optimizing Abstraction
  3 Feb 2024 1 repository listed
  The PseudoInverse (PI) estimator has been introduced to mitigate the variance issue by assuming linearity in the reward function, but this can result in significant bias as this assumption is hard-to-verify from…
• Distributional Off-policy Evaluation with Bellman Residual Minimization
  2 Feb 2024 1 repository listed
  In contrast, we study the more manageable expectation-extended statistical distances and provide a novel theoretical justification on their validity for learning the return distribution.
• RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions
  11 Dec 2023 1 repository listed
  Mobile health leverages personalized and contextually tailored interventions optimized through bandit and reinforcement learning algorithms.
• Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits
  3 Dec 2023 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Off-Policy Evaluation (OPE) in contextual bandits is crucial for assessing new policies using existing data without costly experimentation.
• Towards Assessing and Benchmarking Risk-Return Tradeoff of Off-Policy Evaluation
  30 Nov 2023 1 repository listed
  Existing evaluation metrics for OPE estimators primarily focus on the "accuracy" of OPE or that of downstream policy selection, neglecting risk-return tradeoff in the subsequent online policy deployment.
• SCOPE-RL: A Python Library for Offline Reinforcement Learning and Off-Policy Evaluation
  30 Nov 2023 1 repository listed
  This paper introduces SCOPE-RL, a comprehensive open-source Python software designed for offline reinforcement learning (offline RL), off-policy evaluation (OPE), and selection (OPS).
• When is Off-Policy Evaluation (Reward Modeling) Useful in Contextual Bandits? A Data-Centric Perspective
  23 Nov 2023 1 repository listed Syntology ran 5 of 6 samples · 1 unverified · 6 pointer-only (licence)
  On the one hand, it brings opportunities for safe policy improvement under high-stakes scenarios like clinical guidelines.

Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.