{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tripod-three-complementary-inductive-biases","title":"Tripod: Three Complementary Inductive Biases for Disentangled Representation Learning","arxiv_id":"2404.10282","date":"2024-04-16","proceeding":null,"authors":["Kyle Hsu","Jubayer Ibn Hamid","Kaylee Burns","Chelsea Finn","Jiajun Wu"],"abstract":"Inductive biases are crucial in disentangled representation learning for narrowing down an underspecified solution set. In this work, we consider endowing a neural network autoencoder with three select inductive biases from the literature: data compression into a grid-like latent space via quantization, collective independence amongst latents, and minimal functional influence of any latent on how other latents determine data generation. In principle, these inductive biases are deeply complementary: they most directly specify properties of the latent space, encoder, and decoder, respectively. In practice, however, naively combining existing techniques instantiating these inductive biases fails to yield significant benefits. To address this, we propose adaptations to the three techniques that simplify the learning problem, equip key regularization terms with stabilizing invariances, and quash degenerate incentives. The resulting model, Tripod, achieves state-of-the-art results on a suite of four image disentanglement benchmarks. We also verify that Tripod significantly improves upon its naive incarnation and that all three of its \"legs\" are necessary for best performance.","url_abs":"https://arxiv.org/abs/2404.10282v2","url_pdf":"https://arxiv.org/pdf/2404.10282v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tripod-three-complementary-inductive-biases","repo_url":"https://github.com/kylehkhsu/tripod","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-compression","task_name":"Data Compression"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2404.10282","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.10282"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/kylehkhsu/tripod","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"ran_fixture":1,"unverified":2},"by_repo_kind":{"official":{"samples":9,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0f6edb20446e4d1d","entry":"binary_cross_entropy","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/losses/image.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/losses/image.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f6edb20446e4d1d"}},{"code_sha256_prefix":"68ec2a7d57a8d726","entry":"compute_nmi","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/infomec.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/infomec.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68ec2a7d57a8d726"}},{"code_sha256_prefix":"6900302956eaefd7","entry":"compute_relative_importance_matrix","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/dci.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/dci.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6900302956eaefd7"}},{"code_sha256_prefix":"9eedf1f3e52ffe7e","entry":"disentanglement_per_code","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/dci.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/dci.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9eedf1f3e52ffe7e"}},{"code_sha256_prefix":"1d5c25cbe10e2ded","entry":"mean_squared_error","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/losses/image.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/losses/image.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d5c25cbe10e2ded"}},{"code_sha256_prefix":"3b63b51b8b9d8b8e","entry":"peak_signal_to_noise_ratio","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/image.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/image.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b63b51b8b9d8b8e"}},{"code_sha256_prefix":"37f72845984e17c7","entry":"process_latents","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/dci.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/dci.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"37f72845984e17c7"}},{"code_sha256_prefix":"6011e58708e92cba","entry":"compute_infoe","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/infomec.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/infomec.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6011e58708e92cba"}},{"code_sha256_prefix":"1043f8c5c44231dc","entry":"process_sources","repo":"kylehkhsu/tripod","repo_kind":"official","path":"disentangle/metrics/infomec.py","file_url":"https://github.com/kylehkhsu/tripod/blob/HEAD/disentangle/metrics/infomec.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1043f8c5c44231dc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}