{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/position-aware-self-attention-with-relative","title":"Position-aware Self-attention with Relative Positional Encodings for Slot Filling","arxiv_id":"1807.03052","date":"2018-07-09","proceeding":null,"authors":["Ivan Bilan","Benjamin Roth"],"abstract":"This paper describes how to apply self-attention with relative positional\nencodings to the task of relation extraction. We propose to use the\nself-attention encoder layer together with an additional position-aware\nattention layer that takes into account positions of the query and the object\nin the sentence. The self-attention encoder also uses a custom implementation\nof relative positional encodings which allow each word in the sentence to take\ninto account its left and right context. The evaluation of the model is done on\nthe TACRED dataset. The proposed model relies only on attention (no recurrent\nor convolutional layers are used), while improving performance w.r.t. the\nprevious state of the art.","url_abs":"http://arxiv.org/abs/1807.03052v1","url_pdf":"http://arxiv.org/pdf/1807.03052v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"position-aware-self-attention-with-relative","repo_url":"https://github.com/ivan-bilan/tac-self-attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":null,"task_name":"Position"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"slot-filling","task_name":"Slot Filling"},{"task_slug":"slot-filling-1","task_name":"slot-filling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1807.03052","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}