{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hierarchical-state-abstraction-based-on","title":"Hierarchical State Abstraction Based on Structural Information Principles","arxiv_id":"2304.12000","date":"2023-04-24","proceeding":null,"authors":["Xianghua Zeng","Hao Peng","Angsheng Li","Chunyang Liu","Lifang He","Philip S. Yu"],"abstract":"State abstraction optimizes decision-making by ignoring irrelevant environmental information in reinforcement learning with rich observations. Nevertheless, recent approaches focus on adequate representational capacities resulting in essential information loss, affecting their performances on challenging tasks. In this article, we propose a novel mathematical Structural Information principles-based State Abstraction framework, namely SISA, from the information-theoretic perspective. Specifically, an unsupervised, adaptive hierarchical state clustering method without requiring manual assistance is presented, and meanwhile, an optimal encoding tree is generated. On each non-root tree node, a new aggregation function and condition structural entropy are designed to achieve hierarchical state abstraction and compensate for sampling-induced essential information loss in state abstraction. Empirical evaluations on a visual gridworld domain and six continuous control benchmarks demonstrate that, compared with five SOTA state abstraction approaches, SISA significantly improves mean episode reward and sample efficiency up to 18.98 and 44.44%, respectively. Besides, we experimentally show that SISA is a general framework that can be flexibly integrated with different representation-learning objectives to improve their performances further.","url_abs":"https://arxiv.org/abs/2304.12000v1","url_pdf":"https://arxiv.org/pdf/2304.12000v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hierarchical-state-abstraction-based-on","repo_url":"https://github.com/ringbdstack/sisa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.12000","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.12000"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ringbdstack/sisa","reach":null}],"summary":{"ran_honours":1,"ran":1,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"f5e60b5a2cff1b7e","entry":"CombineDelta","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f5e60b5a2cff1b7e"}},{"code_sha256_prefix":"4796f1f98a1ef807","entry":"PartitionTree","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4796f1f98a1ef807"}},{"code_sha256_prefix":"cb653477a7fefad1","entry":"trans_to_adj","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb653477a7fefad1"}},{"code_sha256_prefix":"1bc1bd0cbe71477b","entry":"update_node","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bc1bd0cbe71477b"}},{"code_sha256_prefix":"90631a4db26644a6","entry":"pool_trans","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"90631a4db26644a6"}},{"code_sha256_prefix":"3624ac45dd420fc6","entry":"trans_to_tree","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3624ac45dd420fc6"}},{"code_sha256_prefix":"064cf08bd1addcc0","entry":"update_depth","repo":"ringbdstack/sisa","repo_kind":"official","path":"source code/dmcontrol/hcse/structural_optimization.py","file_url":"https://github.com/ringbdstack/sisa/blob/HEAD/source%20code/dmcontrol/hcse/structural_optimization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"064cf08bd1addcc0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}