{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/active-vision-reinforcement-learning-under","title":"Active Vision Reinforcement Learning under Limited Visual Observability","arxiv_id":"2306.00975","date":"2023-06-01","proceeding":"NeurIPS 2023 11","authors":["Jinghuan Shang","Michael S. Ryoo"],"abstract":"In this work, we investigate Active Vision Reinforcement Learning (ActiveVision-RL), where an embodied agent simultaneously learns action policy for the task while also controlling its visual observations in partially observable environments. We denote the former as motor policy and the latter as sensory policy. For example, humans solve real world tasks by hand manipulation (motor policy) together with eye movements (sensory policy). ActiveVision-RL poses challenges on coordinating two policies given their mutual influence. We propose SUGARL, Sensorimotor Understanding Guided Active Reinforcement Learning, a framework that models motor and sensory policies separately, but jointly learns them using with an intrinsic sensorimotor reward. This learnable reward is assigned by sensorimotor reward module, incentivizes the sensory policy to select observations that are optimal to infer its own motor action, inspired by the sensorimotor stage of humans. Through a series of experiments, we show the effectiveness of our method across a range of observability conditions and its adaptability to existed RL algorithms. The sensory policies learned through our method are observed to exhibit effective active vision strategies.","url_abs":"https://arxiv.org/abs/2306.00975v2","url_pdf":"https://arxiv.org/pdf/2306.00975v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"active-vision-reinforcement-learning-under","repo_url":"https://github.com/elicassion/sugarl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"active-vision-reinforcement-learning-under","repo_url":"https://github.com/elicassion/active-gym","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.00975","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.00975"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elicassion/active-gym","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elicassion/sugarl","reach":null}],"summary":{"ran_honours":2,"ran":4,"unverified":5},"by_repo_kind":{"official":{"samples":11,"ran":6,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"4c60d8998b722013","entry":"linear_schedule","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/dqn_atari_sugarl.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/dqn_atari_sugarl.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":2,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c60d8998b722013"}},{"code_sha256_prefix":"952670a84f357ba6","entry":"ActiveActor","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"952670a84f357ba6"}},{"code_sha256_prefix":"f57118a5aca08040","entry":"ActiveCritic","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f57118a5aca08040"}},{"code_sha256_prefix":"e39358a081460d4c","entry":"Encoder","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e39358a081460d4c"}},{"code_sha256_prefix":"1a430f9076df383d","entry":"SelfPredictionNetwork","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a430f9076df383d"}},{"code_sha256_prefix":"326b91f6350b35a7","entry":"schedule_drq","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"326b91f6350b35a7"}},{"code_sha256_prefix":"5e69fe653b293ed2","entry":"DrQV2SugarlAgent","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e69fe653b293ed2"}},{"code_sha256_prefix":"25a62c8e4b21cfc0","entry":"FixedFovealEnv","repo":"elicassion/active-gym","repo_kind":"official","path":"active_gym/fov_env.py","file_url":"https://github.com/elicassion/active-gym/blob/HEAD/active_gym/fov_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25a62c8e4b21cfc0"}},{"code_sha256_prefix":"b1e1b5d5f607f350","entry":"FlexibleFovealEnv","repo":"elicassion/active-gym","repo_kind":"official","path":"active_gym/fov_env.py","file_url":"https://github.com/elicassion/active-gym/blob/HEAD/active_gym/fov_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1e1b5d5f607f350"}},{"code_sha256_prefix":"72cc74b1b1979c23","entry":"FlexibleFovealEnvActionType","repo":"elicassion/active-gym","repo_kind":"official","path":"active_gym/fov_env.py","file_url":"https://github.com/elicassion/active-gym/blob/HEAD/active_gym/fov_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"72cc74b1b1979c23"}},{"code_sha256_prefix":"3a2299531e205b0f","entry":"weight_init_drq","repo":"elicassion/sugarl","repo_kind":"official","path":"agent/drqv2_rs_sugarl_sv.py","file_url":"https://github.com/elicassion/sugarl/blob/HEAD/agent/drqv2_rs_sugarl_sv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a2299531e205b0f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}