{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contrastively-disentangled-sequential","title":"Contrastively Disentangled Sequential Variational Autoencoder","arxiv_id":"2110.12091","date":"2021-10-22","proceeding":"NeurIPS 2021 12","authors":["Junwen Bai","Weiran Wang","Carla Gomes"],"abstract":"Self-supervised disentangled representation learning is a critical task in sequence modeling. The learnt representations contribute to better model interpretability as well as the data generation, and improve the sample efficiency for downstream tasks. We propose a novel sequence representation learning method, named Contrastively Disentangled Sequential Variational Autoencoder (C-DSVAE), to extract and separate the static (time-invariant) and dynamic (time-variant) factors in the latent space. Different from previous sequential variational autoencoder methods, we use a novel evidence lower bound which maximizes the mutual information between the input and the latent factors, while penalizes the mutual information between the static and dynamic factors. We leverage contrastive estimations of the mutual information terms in training, together with simple yet effective augmentation techniques, to introduce additional inductive biases. Our experiments show that C-DSVAE significantly outperforms the previous state-of-the-art methods on multiple metrics.","url_abs":"https://arxiv.org/abs/2110.12091v1","url_pdf":"https://arxiv.org/pdf/2110.12091v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contrastively-disentangled-sequential","repo_url":"https://github.com/JunwenBai/C-DSVAE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.12091","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.12091"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JunwenBai/C-DSVAE","reach":null}],"summary":{"ran":6},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"821bc4d6b34cbd45","entry":"CDSVAE","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"821bc4d6b34cbd45"}},{"code_sha256_prefix":"ae91f2cc0db4321d","entry":"LinearUnit","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae91f2cc0db4321d"}},{"code_sha256_prefix":"0488ecf22e3b9241","entry":"conv","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0488ecf22e3b9241"}},{"code_sha256_prefix":"c3f5412779a7544d","entry":"decoder","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3f5412779a7544d"}},{"code_sha256_prefix":"ee3fdcb94a38a1f4","entry":"encoder","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ee3fdcb94a38a1f4"}},{"code_sha256_prefix":"61c7b9ab9b30c9d4","entry":"upconv","repo":"JunwenBai/C-DSVAE","repo_kind":"official","path":"model.py","file_url":"https://github.com/JunwenBai/C-DSVAE/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61c7b9ab9b30c9d4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}