{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mocoda-model-based-counterfactual-data","title":"MoCoDA: Model-based Counterfactual Data Augmentation","arxiv_id":"2210.11287","date":"2022-10-20","proceeding":null,"authors":["Silviu Pitis","Elliot Creager","Ajay Mandlekar","Animesh Garg"],"abstract":"The number of states in a dynamic process is exponential in the number of objects, making reinforcement learning (RL) difficult in complex, multi-object domains. For agents to scale to the real world, they will need to react to and reason about unseen combinations of objects. We argue that the ability to recognize and use local factorization in transition dynamics is a key element in unlocking the power of multi-object reasoning. To this end, we show that (1) known local structure in the environment transitions is sufficient for an exponential reduction in the sample complexity of training a dynamics model, and (2) a locally factored dynamics model provably generalizes out-of-distribution to unseen states and actions. Knowing the local structure also allows us to predict which unseen states and actions this dynamics model will generalize to. We propose to leverage these observations in a novel Model-based Counterfactual Data Augmentation (MoCoDA) framework. MoCoDA applies a learned locally factored dynamics model to an augmented distribution of states and actions to generate counterfactual transitions for RL. MoCoDA works with a broader set of local structures than prior work and allows for direct control over the augmented training distribution. We show that MoCoDA enables RL agents to learn policies that generalize to unseen states and actions. We use MoCoDA to train an offline RL agent to solve an out-of-distribution robotics manipulation task on which standard offline RL algorithms fail.","url_abs":"https://arxiv.org/abs/2210.11287v1","url_pdf":"https://arxiv.org/pdf/2210.11287v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mocoda-model-based-counterfactual-data","repo_url":"https://github.com/spitis/mocoda","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":null,"task_name":"counterfactual"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.11287","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.11287"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/spitis/mrl","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/spitis/mocoda","reach":null}],"summary":{"ran_draft_wrong":5,"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"a35fbd5396153c90","entry":"connected_components","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a35fbd5396153c90"}},{"code_sha256_prefix":"f25cb05933a07e3a","entry":"filter_by_constraint","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f25cb05933a07e3a"}},{"code_sha256_prefix":"215b2276ca908379","entry":"make_graph","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"215b2276ca908379"}},{"code_sha256_prefix":"45fe36afc4e7d025","entry":"naive_root_sampler","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"45fe36afc4e7d025"}},{"code_sha256_prefix":"6a10e7af5d468380","entry":"naive_sampler","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6a10e7af5d468380"}},{"code_sha256_prefix":"b91f2ab6e4300fbe","entry":"split_nodes","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b91f2ab6e4300fbe"}},{"code_sha256_prefix":"8c9f808b47a1ecc3","entry":"generate_gmm_coda_dataset","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8c9f808b47a1ecc3"}},{"code_sha256_prefix":"99b3d0dbe874d55d","entry":"naive_child_sampler","repo":"spitis/mocoda","repo_kind":"official","path":"augment_offline.py","file_url":"https://github.com/spitis/mocoda/blob/HEAD/augment_offline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"99b3d0dbe874d55d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}