{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrastive-representation-for-data-filtering","title":"Contrastive Representation for Data Filtering in Cross-Domain Offline Reinforcement Learning","arxiv_id":"2405.06192","date":"2024-05-10","proceeding":null,"authors":["Xiaoyu Wen","Chenjia Bai","Kang Xu","Xudong Yu","Yang Zhang","Xuelong Li","Zhen Wang"],"abstract":"Cross-domain offline reinforcement learning leverages source domain data with diverse transition dynamics to alleviate the data requirement for the target domain. However, simply merging the data of two domains leads to performance degradation due to the dynamics mismatch. Existing methods address this problem by measuring the dynamics gap via domain classifiers while relying on the assumptions of the transferability of paired domains. In this paper, we propose a novel representation-based approach to measure the domain gap, where the representation is learned through a contrastive objective by sampling transitions from different domains. We show that such an objective recovers the mutual-information gap of transition functions in two domains without suffering from the unbounded issue of the dynamics gap in handling significantly different domains. Based on the representations, we introduce a data filtering algorithm that selectively shares transitions from the source domain according to the contrastive score functions. Empirical results on various tasks demonstrate that our method achieves superior performance, using only 10% of the target data to achieve 89.2% of the performance on 100% target dataset with state-of-the-art methods.","url_abs":"https://arxiv.org/abs/2405.06192v1","url_pdf":"https://arxiv.org/pdf/2405.06192v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrastive-representation-for-data-filtering","repo_url":"https://github.com/battlewen/igdf","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.06192","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.06192"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/battlewen/igdf","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"ran":1,"ran_honours":1,"unverified":4},"by_repo_kind":{"official":{"samples":7,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"639b4986c270df90","entry":"compute_mean_std","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/igdf.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/igdf.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"639b4986c270df90"}},{"code_sha256_prefix":"cef315eeed246be2","entry":"eval_actor","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/igdf_o2o.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/igdf_o2o.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cef315eeed246be2"}},{"code_sha256_prefix":"ba80f9ff81b1de5f","entry":"normalize_states","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/igdf.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/igdf.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba80f9ff81b1de5f"}},{"code_sha256_prefix":"b8653ae9a4e21d3c","entry":"is_terminal_region_for_at","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/env/env_utils.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/env/env_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8653ae9a4e21d3c"}},{"code_sha256_prefix":"b126429395363b00","entry":"is_terminal_region_for_hc","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/env/env_utils.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/env/env_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b126429395363b00"}},{"code_sha256_prefix":"0aa4b9c66abed3d1","entry":"is_terminal_region_for_hp","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/env/env_utils.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/env/env_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0aa4b9c66abed3d1"}},{"code_sha256_prefix":"a8341f5e34d50ae6","entry":"wrap_env","repo":"battlewen/igdf","repo_kind":"official","path":"algorithms/igdf/igdf.py","file_url":"https://github.com/battlewen/igdf/blob/HEAD/algorithms/igdf/igdf.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8341f5e34d50ae6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}