{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/local-metric-learning-for-off-policy","title":"Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions","arxiv_id":"2210.13373","date":"2022-10-24","proceeding":null,"authors":["Haanvid Lee","Jongmin Lee","Yunseon Choi","Wonseok Jeon","Byung-Jun Lee","Yung-Kyun Noh","Kee-Eung Kim"],"abstract":"We consider local kernel metric learning for off-policy evaluation (OPE) of deterministic policies in contextual bandits with continuous action spaces. Our work is motivated by practical scenarios where the target policy needs to be deterministic due to domain requirements, such as prescription of treatment dosage and duration in medicine. Although importance sampling (IS) provides a basic principle for OPE, it is ill-posed for the deterministic target policy with continuous actions. Our main idea is to relax the target policy and pose the problem as kernel-based estimation, where we learn the kernel metric in order to minimize the overall mean squared error (MSE). We present an analytic solution for the optimal metric, based on the analysis of bias and variance. Whereas prior work has been limited to scalar action spaces or kernel bandwidth selection, our work takes a step further being capable of vector action spaces and metric optimization. We show that our estimator is consistent, and significantly reduces the MSE compared to baseline OPE methods through experiments on various domains.","url_abs":"https://arxiv.org/abs/2210.13373v3","url_pdf":"https://arxiv.org/pdf/2210.13373v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"local-metric-learning-for-off-policy","repo_url":"https://github.com/haanvid/kmis","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"},{"task_slug":"off-policy-evaluation","task_name":"Off-policy evaluation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.13373","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.13373"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/haanvid/kmis","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":10},"by_repo_kind":{"official":{"samples":11,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ac63e9a19256d49f","entry":"load_dict","repo":"haanvid/kmis","repo_kind":"official","path":"main_dummy.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/main_dummy.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac63e9a19256d49f"}},{"code_sha256_prefix":"2a6dd5d2668b3f32","entry":"batch_hessian","repo":"haanvid/kmis","repo_kind":"official","path":"util.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2a6dd5d2668b3f32"}},{"code_sha256_prefix":"292ed60be86050df","entry":"batch_hessian_tf","repo":"haanvid/kmis","repo_kind":"official","path":"util.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"292ed60be86050df"}},{"code_sha256_prefix":"57d7d9dc4dc72fd3","entry":"disc_evaluation","repo":"haanvid/kmis","repo_kind":"official","path":"disc_ope.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/disc_ope.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57d7d9dc4dc72fd3"}},{"code_sha256_prefix":"0f48c368746fcb00","entry":"gaussian_kernel","repo":"haanvid/kmis","repo_kind":"official","path":"off_pol_eval_functs.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/off_pol_eval_functs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f48c368746fcb00"}},{"code_sha256_prefix":"7f0b9fa3eb72a958","entry":"gen_unif_state","repo":"haanvid/kmis","repo_kind":"official","path":"main_dummy.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/main_dummy.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7f0b9fa3eb72a958"}},{"code_sha256_prefix":"db802997214da3fe","entry":"get_behavior_policy_prob","repo":"haanvid/kmis","repo_kind":"official","path":"disc_ope.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/disc_ope.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db802997214da3fe"}},{"code_sha256_prefix":"919e956d58f0bc54","entry":"histogram","repo":"haanvid/kmis","repo_kind":"official","path":"disc_ope.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/disc_ope.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"919e956d58f0bc54"}},{"code_sha256_prefix":"a1a22f2643fd92e3","entry":"metric_evaluation","repo":"haanvid/kmis","repo_kind":"official","path":"off_pol_eval_functs.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/off_pol_eval_functs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1a22f2643fd92e3"}},{"code_sha256_prefix":"b562006a4fd9cd97","entry":"off_policy_evaluation","repo":"haanvid/kmis","repo_kind":"official","path":"off_pol_eval_functs.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/off_pol_eval_functs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b562006a4fd9cd97"}},{"code_sha256_prefix":"a66e33d74b337fed","entry":"shuffle_data","repo":"haanvid/kmis","repo_kind":"official","path":"util.py","file_url":"https://github.com/haanvid/kmis/blob/HEAD/util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a66e33d74b337fed"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}