{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stitching-sub-trajectories-with-conditional","title":"Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL","arxiv_id":"2402.07226","date":"2024-02-11","proceeding":null,"authors":["Sungyoon Kim","Yunseon Choi","Daiki E. Matsunaga","Kee-Eung Kim"],"abstract":"Offline Goal-Conditioned Reinforcement Learning (Offline GCRL) is an important problem in RL that focuses on acquiring diverse goal-oriented skills solely from pre-collected behavior datasets. In this setting, the reward feedback is typically absent except when the goal is achieved, which makes it difficult to learn policies especially from a finite dataset of suboptimal behaviors. In addition, realistic scenarios involve long-horizon planning, which necessitates the extraction of useful skills within sub-trajectories. Recently, the conditional diffusion model has been shown to be a promising approach to generate high-quality long-horizon plans for RL. However, their practicality for the goal-conditioned setting is still limited due to a number of technical assumptions made by the methods. In this paper, we propose SSD (Sub-trajectory Stitching with Diffusion), a model-based offline GCRL method that leverages the conditional diffusion model to address these limitations. In summary, we use the diffusion model that generates future plans conditioned on the target goal and value, with the target value estimated from the goal-relabeled offline dataset. We report state-of-the-art performance in the standard benchmark set of GCRL tasks, and demonstrate the capability to successfully stitch the segments of suboptimal trajectories in the offline data to generate high-quality plans.","url_abs":"https://arxiv.org/abs/2402.07226v1","url_pdf":"https://arxiv.org/pdf/2402.07226v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stitching-sub-trajectories-with-conditional","repo_url":"https://github.com/rlatjddbs/ssd","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"offline-rl","task_name":"Offline RL"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"non-maximum-suppression","method_name":"Non Maximum Suppression"},{"method_slug":"set","method_name":"SET"},{"method_slug":"ssd","method_name":"SSD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.07226","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.07226"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rlatjddbs/ssd","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/rlatjddbs/SSD","reach":{"status":"ok"}}],"summary":{"ran":13,"ran_draft_wrong":4,"ran_honours":2,"ran_fixture":1,"unverified":9},"by_repo_kind":{"official":{"samples":29,"ran":20,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":29,"samples":[{"code_sha256_prefix":"7d5fe12da38ea3f5","entry":"Conv1dBlock","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d5fe12da38ea3f5"}},{"code_sha256_prefix":"52a4fe52c83d4acb","entry":"Downsample1d","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52a4fe52c83d4acb"}},{"code_sha256_prefix":"d5a64382ebf227b3","entry":"HindsightCritic","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5a64382ebf227b3"}},{"code_sha256_prefix":"d7aa55fb04049d51","entry":"ResidualTemporalBlock","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d7aa55fb04049d51"}},{"code_sha256_prefix":"92d5319d3f565de2","entry":"Timer","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"92d5319d3f565de2"}},{"code_sha256_prefix":"3ff139e92610787b","entry":"Upsample1d","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3ff139e92610787b"}},{"code_sha256_prefix":"dfe036fdc5d15991","entry":"WeightedL1","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dfe036fdc5d15991"}},{"code_sha256_prefix":"264959cf75b98c11","entry":"WeightedL2","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"264959cf75b98c11"}},{"code_sha256_prefix":"61a25fc7fbbca3a7","entry":"batch_to_device","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61a25fc7fbbca3a7"}},{"code_sha256_prefix":"33fe2742a5ad304c","entry":"cycle","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"33fe2742a5ad304c"}},{"code_sha256_prefix":"1aec056948f8cf9f","entry":"discounted_return","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1aec056948f8cf9f"}},{"code_sha256_prefix":"09e5d94abb73ef7b","entry":"discounted_return","repo":"rlatjddbs/ssd","repo_kind":"official","path":"bc/agents.py","file_url":"https://github.com/rlatjddbs/ssd/blob/HEAD/bc/agents.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09e5d94abb73ef7b"}},{"code_sha256_prefix":"6eb59c36d3f33fe4","entry":"goal_distance","repo":"rlatjddbs/ssd","repo_kind":"official","path":"bc/base_agent.py","file_url":"https://github.com/rlatjddbs/ssd/blob/HEAD/bc/base_agent.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6eb59c36d3f33fe4"}},{"code_sha256_prefix":"67c16af0a0af4b08","entry":"increasing_schedule","repo":"rlatjddbs/ssd","repo_kind":"official","path":"utils/eval_module.py","file_url":"https://github.com/rlatjddbs/ssd/blob/HEAD/utils/eval_module.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67c16af0a0af4b08"}},{"code_sha256_prefix":"77cff4b5b63e486e","entry":"main","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77cff4b5b63e486e"}},{"code_sha256_prefix":"b33aeaa0860fa787","entry":"main_maze","repo":"rlatjddbs/ssd","repo_kind":"official","path":"utils/eval_module.py","file_url":"https://github.com/rlatjddbs/ssd/blob/HEAD/utils/eval_module.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b33aeaa0860fa787"}},{"code_sha256_prefix":"4afff382ab811901","entry":"pad_horizon","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4afff382ab811901"}},{"code_sha256_prefix":"06e523fad286371d","entry":"to_device","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06e523fad286371d"}},{"code_sha256_prefix":"0e8d8251e66ab160","entry":"to_np","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0e8d8251e66ab160"}},{"code_sha256_prefix":"a31d5ee8d54d737b","entry":"to_torch","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a31d5ee8d54d737b"}},{"code_sha256_prefix":"0b6de38ffcf51a1e","entry":"DiffuserCritic","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b6de38ffcf51a1e"}},{"code_sha256_prefix":"880a368bc0c23611","entry":"GaussianDiffusion","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"880a368bc0c23611"}},{"code_sha256_prefix":"0d1e4f14a279e96a","entry":"TemporalUnetTransformer","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d1e4f14a279e96a"}},{"code_sha256_prefix":"ddcd47f61ee383c7","entry":"ValueL1","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ddcd47f61ee383c7"}},{"code_sha256_prefix":"14519433f6d88b11","entry":"ValueL2","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"14519433f6d88b11"}},{"code_sha256_prefix":"ab29d5d735d5a9cb","entry":"ValueLoss","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ab29d5d735d5a9cb"}},{"code_sha256_prefix":"aa6bfea7b5d980c8","entry":"WeightedLoss","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa6bfea7b5d980c8"}},{"code_sha256_prefix":"26a7c332d2d6fdbb","entry":"copy_nn_module","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26a7c332d2d6fdbb"}},{"code_sha256_prefix":"49af4ec227a55bc8","entry":"soft_copy_nn_module","repo":"rlatjddbs/SSD","repo_kind":"official","path":"dc/dc.py","file_url":"https://github.com/rlatjddbs/SSD/blob/HEAD/dc/dc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"49af4ec227a55bc8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}