{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/local-disentanglement-in-variational-auto","title":"Local Disentanglement in Variational Auto-Encoders Using Jacobian $L_1$ Regularization","arxiv_id":"2106.02923","date":"2021-06-05","proceeding":"NeurIPS 2021 12","authors":["Travers Rhodes","Daniel D. Lee"],"abstract":"There have been many recent advances in representation learning; however, unsupervised representation learning can still struggle with model identification issues related to rotations of the latent space. Variational Auto-Encoders (VAEs) and their extensions such as $\\beta$-VAEs have been shown to improve local alignment of latent variables with PCA directions, which can help to improve model disentanglement under some conditions. Borrowing inspiration from Independent Component Analysis (ICA) and sparse coding, we propose applying an $L_1$ loss to the VAE's generative Jacobian during training to encourage local latent variable alignment with independent factors of variation in images of multiple objects or images with multiple parts. We demonstrate our results on a variety of datasets, giving qualitative and quantitative results using information theoretic and modularity measures that show our added $L_1$ cost encourages local axis alignment of the latent representation with individual factors of variation.","url_abs":"https://arxiv.org/abs/2106.02923v2","url_pdf":"https://arxiv.org/pdf/2106.02923v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"local-disentanglement-in-variational-auto","repo_url":"https://github.com/travers-rhodes/jlonevae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"pca","method_name":"PCA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.02923","atlas_url":"https://app.syntology.ai/?focus=2106.02923","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.02923"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/travers-rhodes/jlonevae","reach":null}],"summary":{"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"36bd19e41d6d0203","entry":"compute_generator_jacobian_optimized","repo":"travers-rhodes/jlonevae","repo_kind":"official","path":"jlonevae_lib/architecture/vae_jacobian.py","file_url":"https://github.com/travers-rhodes/jlonevae/blob/HEAD/jlonevae_lib/architecture/vae_jacobian.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"36bd19e41d6d0203"}},{"code_sha256_prefix":"a396932d7dc104c9","entry":"jacobian_loss_function","repo":"travers-rhodes/jlonevae","repo_kind":"official","path":"jlonevae_lib/architecture/vae_jacobian.py","file_url":"https://github.com/travers-rhodes/jlonevae/blob/HEAD/jlonevae_lib/architecture/vae_jacobian.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a396932d7dc104c9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}