{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seqdiffuseq-text-diffusion-with-encoder","title":"SeqDiffuSeq: Text Diffusion with Encoder-Decoder Transformers","arxiv_id":"2212.10325","date":"2022-12-20","proceeding":null,"authors":["Hongyi Yuan","Zheng Yuan","Chuanqi Tan","Fei Huang","Songfang Huang"],"abstract":"Diffusion model, a new generative modelling paradigm, has achieved great success in image, audio, and video generation. However, considering the discrete categorical nature of text, it is not trivial to extend continuous diffusion models to natural language, and text diffusion models are less studied. Sequence-to-sequence text generation is one of the essential natural language processing topics. In this work, we apply diffusion models to approach sequence-to-sequence text generation, and explore whether the superiority generation performance of diffusion model can transfer to natural language domain. We propose SeqDiffuSeq, a text diffusion model for sequence-to-sequence generation. SeqDiffuSeq uses an encoder-decoder Transformers architecture to model denoising function. In order to improve generation quality, SeqDiffuSeq combines the self-conditioning technique and a newly proposed adaptive noise schedule technique. The adaptive noise schedule has the difficulty of denoising evenly distributed across time steps, and considers exclusive noise schedules for tokens at different positional order. Experiment results illustrate the good performance on sequence-to-sequence generation in terms of text quality and inference time.","url_abs":"https://arxiv.org/abs/2212.10325v5","url_pdf":"https://arxiv.org/pdf/2212.10325v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seqdiffuseq-text-diffusion-with-encoder","repo_url":"https://github.com/yuanhy1997/seqdiffuseq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.10325","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.10325"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuanhy1997/seqdiffuseq","reach":null}],"summary":{"ran_honours":3},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"2ab2316ac6fdd869","entry":"betas_for_alpha_bar","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"2ab2316ac6fdd869"}},{"code_sha256_prefix":"bcdd9b34f09283d7","entry":"betas_for_alpha_bar_trunc_cosine","repo":"yuanhy1997/seqdiffuseq","repo_kind":"official","path":"src/modeling/diffusion/gaussian_diffusion.py","file_url":"https://github.com/yuanhy1997/seqdiffuseq/blob/HEAD/src/modeling/diffusion/gaussian_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bcdd9b34f09283d7"}},{"code_sha256_prefix":"0d06a031b51dea9a","entry":"get_named_beta_schedule","repo":"yuanhy1997/seqdiffuseq","repo_kind":"official","path":"src/modeling/diffusion/gaussian_diffusion.py","file_url":"https://github.com/yuanhy1997/seqdiffuseq/blob/HEAD/src/modeling/diffusion/gaussian_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d06a031b51dea9a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}