{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/improving-variational-autoencoder-for-text","title":"Improving Variational Autoencoder for Text Modelling with Timestep-Wise Regularisation","arxiv_id":"2011.01136","date":"2020-11-02","proceeding":"COLING 2020 8","authors":["Ruizhe Li","Xiao Li","Guanyi Chen","Chenghua Lin"],"abstract":"The Variational Autoencoder (VAE) is a popular and powerful model applied to text modelling to generate diverse sentences. However, an issue known as posterior collapse (or KL loss vanishing) happens when the VAE is used in text modelling, where the approximate posterior collapses to the prior, and the model will totally ignore the latent variables and be degraded to a plain language model during text generation. Such an issue is particularly prevalent when RNN-based VAE models are employed for text modelling. In this paper, we propose a simple, generic architecture called Timestep-Wise Regularisation VAE (TWR-VAE), which can effectively avoid posterior collapse and can be applied to any RNN-based VAE models. The effectiveness and versatility of our model are demonstrated in different tasks, including language modelling and dialogue response generation.","url_abs":"https://arxiv.org/abs/2011.01136v2","url_pdf":"https://arxiv.org/pdf/2011.01136v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"improving-variational-autoencoder-for-text","repo_url":"https://github.com/ruizheliUOA/TWR-VAE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"response-generation","task_name":"Response Generation"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2011.01136","atlas_url":"https://app.syntology.ai/?focus=2011.01136","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2011.01136"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruizheliUOA/TWR-VAE","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e8f5c9d65e322509","entry":"gData","repo":"ruizheliUOA/TWR-VAE","repo_kind":"official","path":"dialogue_dd/utils_dailydial.py","file_url":"https://github.com/ruizheliUOA/TWR-VAE/blob/HEAD/dialogue_dd/utils_dailydial.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8f5c9d65e322509"}},{"code_sha256_prefix":"ea265e7b95742053","entry":"indexes2sent","repo":"ruizheliUOA/TWR-VAE","repo_kind":"official","path":"dialogue_dd/utils_dailydial.py","file_url":"https://github.com/ruizheliUOA/TWR-VAE/blob/HEAD/dialogue_dd/utils_dailydial.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea265e7b95742053"}},{"code_sha256_prefix":"3c6217cf88649dbf","entry":"sent2indexes","repo":"ruizheliUOA/TWR-VAE","repo_kind":"official","path":"dialogue_dd/utils_dailydial.py","file_url":"https://github.com/ruizheliUOA/TWR-VAE/blob/HEAD/dialogue_dd/utils_dailydial.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c6217cf88649dbf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}