{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/from-masked-language-modeling-to-translation","title":"From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding","arxiv_id":"2105.07316","date":"2021-05-15","proceeding":"NAACL 2021 4","authors":["Rob van der Goot","Ibrahim Sharaf","Aizhan Imankulova","Ahmet Üstün","Marija Stepanović","Alan Ramponi","Siti Oryza Khairunnisa","Mamoru Komachi","Barbara Plank"],"abstract":"The lack of publicly available evaluation data for low-resource languages limits progress in Spoken Language Understanding (SLU). As key tasks like intent classification and slot filling require abundant training data, it is desirable to reuse existing data in high-resource languages to develop models for low-resource scenarios. We introduce xSID, a new benchmark for cross-lingual Slot and Intent Detection in 13 languages from 6 language families, including a very low-resource dialect. To tackle the challenge, we propose a joint learning approach, with English SLU training data and non-English auxiliary tasks from raw text, syntax and translation for transfer. We study two setups which differ by type and language coverage of the pre-trained embeddings. Our results show that jointly learning the main tasks with masked language modeling is effective for slots, while machine translation transfer works best for intent classification.","url_abs":"https://arxiv.org/abs/2105.07316v1","url_pdf":"https://arxiv.org/pdf/2105.07316v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"from-masked-language-modeling-to-translation","repo_url":"https://bitbucket.org/robvanderg/xsid","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"from-masked-language-modeling-to-translation","repo_url":"https://github.com/Kaleidophon/deep-significance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"intent-classification","task_name":"Intent Classification"},{"task_slug":"intent-detection","task_name":"Intent Detection"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"masked-language-modeling","task_name":"Masked Language Modeling"},{"task_slug":"slot-filling","task_name":"Slot Filling"},{"task_slug":"spoken-language-understanding","task_name":"Spoken Language Understanding"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"intent-classification-1","task_name":"intent-classification"}],"methods":[],"datasets_introduced":[{"slug":"xsid","name":"xSID","full_name":"Cross-lingual Slot and Intent Detection"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.07316","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}