{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/riemannian-normalizing-flow-on-variational","title":"Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling","arxiv_id":"1904.02399","date":"2019-04-04","proceeding":"NAACL 2019 6","authors":["Prince Zizhuang Wang","William Yang Wang"],"abstract":"Recurrent Variational Autoencoder has been widely used for language modeling\nand text generation tasks. These models often face a difficult optimization\nproblem, also known as the Kullback-Leibler (KL) term vanishing issue, where\nthe posterior easily collapses to the prior, and the model will ignore latent\ncodes in generative tasks. To address this problem, we introduce an improved\nWasserstein Variational Autoencoder (WAE) with Riemannian Normalizing Flow\n(RNF) for text modeling. The RNF transforms a latent variable into a space that\nrespects the geometric characteristics of input space, which makes posterior\nimpossible to collapse to the non-informative prior. The Wasserstein objective\nminimizes the distance between the marginal distribution and the prior directly\nand therefore does not force the posterior to match the prior. Empirical\nexperiments show that our model avoids KL vanishing over a range of datasets\nand has better performances in tasks such as language modeling, likelihood\napproximation, and text generation. Through a series of experiments and\nanalysis over latent space, we show that our model learns latent distributions\nthat respect latent space geometry and is able to generate sentences that are\nmore diverse.","url_abs":"http://arxiv.org/abs/1904.02399v4","url_pdf":"http://arxiv.org/pdf/1904.02399v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"riemannian-normalizing-flow-on-variational","repo_url":"https://github.com/kingofspace0wzz/wae-rnf-lm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1904.02399","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.02399"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kingofspace0wzz/wae-rnf-lm","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":13},"by_repo_kind":{"official":{"samples":14,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"83d1c2e11d595efa","entry":"detach","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"copula-lm/model.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/copula-lm/model.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"83d1c2e11d595efa"}},{"code_sha256_prefix":"05ee14317de0f42a","entry":"bow_recon_loss","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"copula-lm/loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/copula-lm/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05ee14317de0f42a"}},{"code_sha256_prefix":"6ad6d375bcd296f1","entry":"flow_kld","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ad6d375bcd296f1"}},{"code_sha256_prefix":"309c3db1dd9740dc","entry":"forgy","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"clustering/kmeans.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/clustering/kmeans.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"309c3db1dd9740dc"}},{"code_sha256_prefix":"b067a64447478f3f","entry":"group_pairwise","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"clustering/pairwise.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/clustering/pairwise.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b067a64447478f3f"}},{"code_sha256_prefix":"493fdeb2279b439e","entry":"im_metric","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"stochastic/flow.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/stochastic/flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"493fdeb2279b439e"}},{"code_sha256_prefix":"7b95c648cb0fb5f2","entry":"log_sum_exp","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"model.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b95c648cb0fb5f2"}},{"code_sha256_prefix":"ca442985ac3b7e16","entry":"metric","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"stochastic/flow.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/stochastic/flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca442985ac3b7e16"}},{"code_sha256_prefix":"b67c0c61b3e58c05","entry":"pairwise_distance","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"clustering/pairwise.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/clustering/pairwise.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b67c0c61b3e58c05"}},{"code_sha256_prefix":"f78950c60af411a7","entry":"recon_loss","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f78950c60af411a7"}},{"code_sha256_prefix":"06daa8767944cf66","entry":"seq_recon_loss","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"copula-lm/loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/copula-lm/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06daa8767944cf66"}},{"code_sha256_prefix":"1bb0df71b00e39af","entry":"square_norm","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"stochastic/flow.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/stochastic/flow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bb0df71b00e39af"}},{"code_sha256_prefix":"e80403ae8dfd6e68","entry":"total_kld","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e80403ae8dfd6e68"}},{"code_sha256_prefix":"405fac421afa9586","entry":"total_kld","repo":"kingofspace0wzz/wae-rnf-lm","repo_kind":"official","path":"copula-lm/loss.py","file_url":"https://github.com/kingofspace0wzz/wae-rnf-lm/blob/HEAD/copula-lm/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"405fac421afa9586"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}