{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/marginal-density-ratio-for-off-policy-1","title":"Marginal Density Ratio for Off-Policy Evaluation in Contextual Bandits","arxiv_id":"2312.01457","date":"2023-12-03","proceeding":"NeurIPS 2023 11","authors":["Muhammad Faaiz Taufiq","Arnaud Doucet","Rob Cornish","Jean-Francois Ton"],"abstract":"Off-Policy Evaluation (OPE) in contextual bandits is crucial for assessing new policies using existing data without costly experimentation. However, current OPE methods, such as Inverse Probability Weighting (IPW) and Doubly Robust (DR) estimators, suffer from high variance, particularly in cases of low overlap between target and behavior policies or large action and context spaces. In this paper, we introduce a new OPE estimator for contextual bandits, the Marginal Ratio (MR) estimator, which focuses on the shift in the marginal distribution of outcomes $Y$ instead of the policies themselves. Through rigorous theoretical analysis, we demonstrate the benefits of the MR estimator compared to conventional methods like IPW and DR in terms of variance reduction. Additionally, we establish a connection between the MR estimator and the state-of-the-art Marginalized Inverse Propensity Score (MIPS) estimator, proving that MR achieves lower variance among a generalized family of MIPS estimators. We further illustrate the utility of the MR estimator in causal inference settings, where it exhibits enhanced performance in estimating Average Treatment Effects (ATE). Our experiments on synthetic and real-world datasets corroborate our theoretical findings and highlight the practical advantages of the MR estimator in OPE for contextual bandits.","url_abs":"https://arxiv.org/abs/2312.01457v1","url_pdf":"https://arxiv.org/pdf/2312.01457v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"marginal-density-ratio-for-off-policy-1","repo_url":"https://github.com/faaizt/mr-ope","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"causal-inference","task_name":"Causal Inference"},{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"},{"task_slug":"off-policy-evaluation","task_name":"Off-policy evaluation"}],"methods":[{"method_slug":"causal-inference","method_name":"Causal inference"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.01457","atlas_url":"https://app.syntology.ai/?focus=2312.01457","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.01457"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/faaizT/MR-OPE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/faaizt/mr-ope","reach":{"status":"ok"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"d0a0a2e6d4e887c8","entry":"gen_eps_greedy","repo":"faaizT/MR-OPE","repo_kind":"official","path":"src/mips_experiment_utils/policy.py","file_url":"https://github.com/faaizT/MR-OPE/blob/HEAD/src/mips_experiment_utils/policy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0a0a2e6d4e887c8"}},{"code_sha256_prefix":"bc202756013711a0","entry":"load_data","repo":"faaizT/MR-OPE","repo_kind":"official","path":"src/obp_utils/data_loader_funcs.py","file_url":"https://github.com/faaizT/MR-OPE/blob/HEAD/src/obp_utils/data_loader_funcs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bc202756013711a0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}