{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sentence-embedding-alignment-for-lifelong","title":"Sentence Embedding Alignment for Lifelong Relation Extraction","arxiv_id":"1903.02588","date":"2019-03-06","proceeding":"NAACL 2019 6","authors":["Hong Wang","Wenhan Xiong","Mo Yu","Xiaoxiao Guo","Shiyu Chang","William Yang Wang"],"abstract":"Conventional approaches to relation extraction usually require a fixed set of\npre-defined relations. Such requirement is hard to meet in many real\napplications, especially when new data and relations are emerging incessantly\nand it is computationally expensive to store all data and re-train the whole\nmodel every time new data and relations come in. We formulate such a\nchallenging problem as lifelong relation extraction and investigate\nmemory-efficient incremental learning methods without catastrophically\nforgetting knowledge learned from previous tasks. We first investigate a\nmodified version of the stochastic gradient methods with a replay memory, which\nsurprisingly outperforms recent state-of-the-art lifelong learning methods. We\nfurther propose to improve this approach to alleviate the forgetting problem by\nanchoring the sentence embedding space. Specifically, we utilize an explicit\nalignment model to mitigate the sentence embedding distortion of the learned\nmodel when training on new data and new relations. Experiment results on\nmultiple benchmarks show that our proposed method significantly outperforms the\nstate-of-the-art lifelong learning approaches.","url_abs":"http://arxiv.org/abs/1903.02588v3","url_pdf":"http://arxiv.org/pdf/1903.02588v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sentence-embedding-alignment-for-lifelong","repo_url":"https://github.com/hongwang600/Lifelong_Relation_Detection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"sentence-embedding-alignment-for-lifelong","repo_url":"https://github.com/wutong8023/Lifelong_Relation_Detection_with_Meta_Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"incremental-learning","task_name":"Incremental Learning"},{"task_slug":"lifelong-learning","task_name":"Lifelong learning"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-embedding","task_name":"Sentence Embedding"},{"task_slug":"sentence-embedding-1","task_name":"Sentence-Embedding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.02588","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}