{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/proper-laplacian-representation-learning","title":"Proper Laplacian Representation Learning","arxiv_id":"2310.10833","date":"2023-10-16","proceeding":null,"authors":["Diego Gomez","Michael Bowling","Marlos C. Machado"],"abstract":"The ability to learn good representations of states is essential for solving large reinforcement learning problems, where exploration, generalization, and transfer are particularly challenging. The Laplacian representation is a promising approach to address these problems by inducing informative state encoding and intrinsic rewards for temporally-extended action discovery and reward shaping. To obtain the Laplacian representation one needs to compute the eigensystem of the graph Laplacian, which is often approximated through optimization objectives compatible with deep learning approaches. These approximations, however, depend on hyperparameters that are impossible to tune efficiently, converge to arbitrary rotations of the desired eigenvectors, and are unable to accurately recover the corresponding eigenvalues. In this paper we introduce a theoretically sound objective and corresponding optimization algorithm for approximating the Laplacian representation. Our approach naturally recovers both the true eigenvectors and eigenvalues while eliminating the hyperparameter dependence of previous approximations. We provide theoretical guarantees for our method and we show that those results translate empirically into robust learning across multiple environments.","url_abs":"https://arxiv.org/abs/2310.10833v2","url_pdf":"https://arxiv.org/pdf/2310.10833v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"proper-laplacian-representation-learning","repo_url":"https://github.com/tarod13/laplacian_dual_dynamics","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"proper-laplacian-representation-learning","repo_url":"https://github.com/mklissa/dceo","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.10833","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.10833"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mklissa/dceo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tarod13/laplacian_dual_dynamics","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":2,"ran":4,"unverified":3},"by_repo_kind":{"official":{"samples":7,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"bbebc7418e5cfd5c","entry":"act","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"bbebc7418e5cfd5c"}},{"code_sha256_prefix":"272f03113fb1f4b5","entry":"discounted_sampling","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/agent/episodic_replay_buffer.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/agent/episodic_replay_buffer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"272f03113fb1f4b5"}},{"code_sha256_prefix":"d2c59e50d28a6697","entry":"generate_hk_get_variables_fn","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/nets/utils.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2c59e50d28a6697"}},{"code_sha256_prefix":"04047b9930bcdabb","entry":"generate_hk_module_fn","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/nets/utils.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"04047b9930bcdabb"}},{"code_sha256_prefix":"2957ca0c26f2fc3f","entry":"select_action","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2957ca0c26f2fc3f"}},{"code_sha256_prefix":"d563a2a842177015","entry":"uniform_sampling","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/agent/episodic_replay_buffer.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/agent/episodic_replay_buffer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d563a2a842177015"}},{"code_sha256_prefix":"8efb0fb2de2eab05","entry":"generate_conv_layers","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/nets/mlp.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/nets/mlp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8efb0fb2de2eab05"}},{"code_sha256_prefix":"1b345642da63f487","entry":"generate_fc_layers","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/nets/mlp.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/nets/mlp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b345642da63f487"}},{"code_sha256_prefix":"df14a06abf4242f7","entry":"update_params","repo":"tarod13/laplacian_dual_dynamics","repo_kind":"official","path":"src/nets/utils.py","file_url":"https://github.com/tarod13/laplacian_dual_dynamics/blob/HEAD/src/nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df14a06abf4242f7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}