{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-far-i-ll-go-offline-goal-conditioned","title":"How Far I'll Go: Offline Goal-Conditioned Reinforcement Learning via $f$-Advantage Regression","arxiv_id":"2206.03023","date":"2022-06-07","proceeding":null,"authors":["Yecheng Jason Ma","Jason Yan","Dinesh Jayaraman","Osbert Bastani"],"abstract":"Offline goal-conditioned reinforcement learning (GCRL) promises general-purpose skill learning in the form of reaching diverse goals from purely offline datasets. We propose $\\textbf{Go}$al-conditioned $f$-$\\textbf{A}$dvantage $\\textbf{R}$egression (GoFAR), a novel regression-based offline GCRL algorithm derived from a state-occupancy matching perspective; the key intuition is that the goal-reaching task can be formulated as a state-occupancy matching problem between a dynamics-abiding imitator agent and an expert agent that directly teleports to the goal. In contrast to prior approaches, GoFAR does not require any hindsight relabeling and enjoys uninterleaved optimization for its value and policy networks. These distinct features confer GoFAR with much better offline performance and stability as well as statistical performance guarantee that is unattainable for prior methods. Furthermore, we demonstrate that GoFAR's training objectives can be re-purposed to learn an agent-independent goal-conditioned planner from purely offline source-domain data, which enables zero-shot transfer to new target domains. Through extensive experiments, we validate GoFAR's effectiveness in various problem settings and tasks, significantly outperforming prior state-of-art. Notably, on a real robotic dexterous manipulation task, while no other method makes meaningful progress, GoFAR acquires complex manipulation behavior that successfully accomplishes diverse goals.","url_abs":"https://arxiv.org/abs/2206.03023v2","url_pdf":"https://arxiv.org/pdf/2206.03023v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-far-i-ll-go-offline-goal-conditioned","repo_url":"https://github.com/jasonma2016/gofar","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"regression-1","task_name":"regression"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.03023","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.03023"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jasonma2016/gofar","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran":4},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cf5ce98955ab4534","entry":"atanh","repo":"jasonma2016/gofar","repo_kind":"official","path":"rl_modules/models.py","file_url":"https://github.com/jasonma2016/gofar/blob/HEAD/rl_modules/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf5ce98955ab4534"}},{"code_sha256_prefix":"f395bfda2e4813d7","entry":"create_stats_ordered_dict","repo":"jasonma2016/gofar","repo_kind":"official","path":"envs/env_util.py","file_url":"https://github.com/jasonma2016/gofar/blob/HEAD/envs/env_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f395bfda2e4813d7"}},{"code_sha256_prefix":"a5d10a0b704c8db7","entry":"get_average_returns","repo":"jasonma2016/gofar","repo_kind":"official","path":"envs/env_util.py","file_url":"https://github.com/jasonma2016/gofar/blob/HEAD/envs/env_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a5d10a0b704c8db7"}},{"code_sha256_prefix":"c809703793017cd2","entry":"get_generic_path_information","repo":"jasonma2016/gofar","repo_kind":"official","path":"envs/env_util.py","file_url":"https://github.com/jasonma2016/gofar/blob/HEAD/envs/env_util.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c809703793017cd2"}},{"code_sha256_prefix":"6eb59c36d3f33fe4","entry":"goal_distance","repo":"jasonma2016/gofar","repo_kind":"official","path":"rl_modules/base_agent.py","file_url":"https://github.com/jasonma2016/gofar/blob/HEAD/rl_modules/base_agent.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6eb59c36d3f33fe4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}