{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variational-attention-for-sequence-to","title":"Variational Attention for Sequence-to-Sequence Models","arxiv_id":"1712.08207","date":"2017-12-21","proceeding":"COLING 2018 8","authors":["Hareesh Bahuleyan","Lili Mou","Olga Vechtomova","Pascal Poupart"],"abstract":"The variational encoder-decoder (VED) encodes source information as a set of\nrandom variables using a neural network, which in turn is decoded into target\ndata using another neural network. In natural language processing,\nsequence-to-sequence (Seq2Seq) models typically serve as encoder-decoder\nnetworks. When combined with a traditional (deterministic) attention mechanism,\nthe variational latent space may be bypassed by the attention model, and thus\nbecomes ineffective. In this paper, we propose a variational attention\nmechanism for VED, where the attention vector is also modeled as Gaussian\ndistributed random variables. Results on two experiments show that, without\nloss of quality, our proposed method alleviates the bypassing phenomenon as it\nincreases the diversity of generated sentences.","url_abs":"http://arxiv.org/abs/1712.08207v3","url_pdf":"http://arxiv.org/pdf/1712.08207v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variational-attention-for-sequence-to","repo_url":"https://github.com/HareeshBahuleyan/tf-var-attention","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"variational-attention-for-sequence-to","repo_url":"https://github.com/keshavvinayak01/Dramatic-Chatbot","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"diversity","task_name":"Diversity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1712.08207","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1712.08207"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/HareeshBahuleyan/tf-var-attention","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keshavvinayak01/Dramatic-Chatbot","reach":{"status":"unanswered"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b630e91286e31688","entry":"load_data","repo":"HareeshBahuleyan/tf-var-attention","repo_kind":"official","path":"w2v_generator.py","file_url":"https://github.com/HareeshBahuleyan/tf-var-attention/blob/HEAD/w2v_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b630e91286e31688"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}