{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visual-imitation-learning-with-patch-rewards","title":"Visual Imitation Learning with Patch Rewards","arxiv_id":"2302.00965","date":"2023-02-02","proceeding":null,"authors":["Minghuan Liu","Tairan He","Weinan Zhang","Shuicheng Yan","Zhongwen Xu"],"abstract":"Visual imitation learning enables reinforcement learning agents to learn to behave from expert visual demonstrations such as videos or image sequences, without explicit, well-defined rewards. Previous research either adopted supervised learning techniques or induce simple and coarse scalar rewards from pixels, neglecting the dense information contained in the image demonstrations. In this work, we propose to measure the expertise of various local regions of image samples, or called \\textit{patches}, and recover multi-dimensional \\textit{patch rewards} accordingly. Patch reward is a more precise rewarding characterization that serves as a fine-grained expertise measurement and visual explainability tool. Specifically, we present Adversarial Imitation Learning with Patch Rewards (PatchAIL), which employs a patch-based discriminator to measure the expertise of different local parts from given images and provide patch rewards. The patch-based knowledge is also used to regularize the aggregated reward and stabilize the training. We evaluate our method on DeepMind Control Suite and Atari tasks. The experiment results have demonstrated that PatchAIL outperforms baseline methods and provides valuable interpretations for visual demonstrations.","url_abs":"https://arxiv.org/abs/2302.00965v2","url_pdf":"https://arxiv.org/pdf/2302.00965v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visual-imitation-learning-with-patch-rewards","repo_url":"https://github.com/sail-sg/patchail","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"imitation-learning","task_name":"Imitation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2302.00965","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2302.00965"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/sail-sg/PatchAIL","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sail-sg/patchail","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"ran":2,"ran_honours":2,"ran_draft_wrong":1,"unverified":5},"by_repo_kind":{"official":{"samples":11,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3201fb6cb44890b3","entry":"cosine_distance","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/rewarder.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/rewarder.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3201fb6cb44890b3"}},{"code_sha256_prefix":"9f840a22d31a89bc","entry":"episode_len","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/replay_buffer.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/replay_buffer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9f840a22d31a89bc"}},{"code_sha256_prefix":"5ddff5153e2b8494","entry":"euclidean_distance","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/rewarder.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/rewarder.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ddff5153e2b8494"}},{"code_sha256_prefix":"184876e5f4ab6562","entry":"load_episode","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/replay_buffer.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/replay_buffer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"184876e5f4ab6562"}},{"code_sha256_prefix":"ae234269debed3f4","entry":"schedule","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/utils.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae234269debed3f4"}},{"code_sha256_prefix":"6244e9922f4e0610","entry":"to_torch","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/utils.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6244e9922f4e0610"}},{"code_sha256_prefix":"d7749f49287c02d5","entry":"compute_gradient_penalty","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/agent/adv_irl.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/agent/adv_irl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d7749f49287c02d5"}},{"code_sha256_prefix":"08c2a94185babfce","entry":"make_agent","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/generate.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"08c2a94185babfce"}},{"code_sha256_prefix":"69d19134097dd51c","entry":"make_replay_loader","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/replay_buffer.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/replay_buffer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"69d19134097dd51c"}},{"code_sha256_prefix":"d1fbf3515dde771e","entry":"optimal_transport_plan","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/rewarder.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/rewarder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1fbf3515dde771e"}},{"code_sha256_prefix":"19a25f21c8bbd6b7","entry":"update_mean_var_count_from_moments","repo":"sail-sg/PatchAIL","repo_kind":"official","path":"PatchAIL/utils.py","file_url":"https://github.com/sail-sg/PatchAIL/blob/HEAD/PatchAIL/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19a25f21c8bbd6b7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}