{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zmbart-an-unsupervised-cross-lingual-transfer","title":"ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation","arxiv_id":"2106.01597","date":"2021-06-03","proceeding":"Findings (ACL) 2021 8","authors":["Kaushal Kumar Maurya","Maunendra Sankar Desarkar","Yoshinobu Kano","Kumari Deepshikha"],"abstract":"Despite the recent advancement in NLP research, cross-lingual transfer for natural language generation is relatively understudied. In this work, we transfer supervision from high resource language (HRL) to multiple low-resource languages (LRLs) for natural language generation (NLG). We consider four NLG tasks (text summarization, question generation, news headline generation, and distractor generation) and three syntactically diverse languages, i.e., English, Hindi, and Japanese. We propose an unsupervised cross-lingual language generation framework (called ZmBART) that does not use any parallel or pseudo-parallel/back-translated data. In this framework, we further pre-train mBART sequence-to-sequence denoising auto-encoder model with an auxiliary task using monolingual data of three languages. The objective function of the auxiliary task is close to the target tasks which enriches the multi-lingual latent representation of mBART and provides good initialization for target tasks. Then, this model is fine-tuned with task-specific supervised English data and directly evaluated with low-resource languages in the Zero-shot setting. To overcome catastrophic forgetting and spurious correlation issues, we applied freezing model component and data argumentation approaches respectively. This simple modeling approach gave us promising results.We experimented with few-shot training (with 1000 supervised data points) which boosted the model performance further. We performed several ablations and cross-lingual transferability analyses to demonstrate the robustness of ZmBART.","url_abs":"https://arxiv.org/abs/2106.01597v1","url_pdf":"https://arxiv.org/pdf/2106.01597v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zmbart-an-unsupervised-cross-lingual-transfer","repo_url":"https://github.com/kaushal0494/ZmBART","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"cross-lingual-transfer","task_name":"Cross-Lingual Transfer"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"distractor-generation","task_name":"Distractor Generation"},{"task_slug":"headline-generation","task_name":"Headline Generation"},{"task_slug":"question-generation","task_name":"Question Generation"},{"task_slug":"question-generation","task_name":"Question-Generation"},{"task_slug":"text-generation","task_name":"Text Generation"},{"task_slug":"text-summarization","task_name":"Text Summarization"}],"methods":[{"method_slug":"mbart","method_name":"mBART"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.01597","atlas_url":"https://app.syntology.ai/?focus=2106.01597","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.01597"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaushal0494/ZmBART","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":1},"by_repo_kind":{"official":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"07969f7041022050","entry":"distributed_init","repo":"kaushal0494/ZmBART","repo_kind":"official","path":"fairseq/distributed_utils.py","file_url":"https://github.com/kaushal0494/ZmBART/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"07969f7041022050"}},{"code_sha256_prefix":"90b79fe15491de83","entry":"is_master","repo":"kaushal0494/ZmBART","repo_kind":"official","path":"fairseq/distributed_utils.py","file_url":"https://github.com/kaushal0494/ZmBART/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"90b79fe15491de83"}},{"code_sha256_prefix":"82bdf940665aeeb4","entry":"safe_readline","repo":"kaushal0494/ZmBART","repo_kind":"official","path":"fairseq/binarizer.py","file_url":"https://github.com/kaushal0494/ZmBART/blob/HEAD/fairseq/binarizer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"82bdf940665aeeb4"}},{"code_sha256_prefix":"199e96db5edd0edd","entry":"all_reduce","repo":"kaushal0494/ZmBART","repo_kind":"official","path":"fairseq/distributed_utils.py","file_url":"https://github.com/kaushal0494/ZmBART/blob/HEAD/fairseq/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"199e96db5edd0edd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}