{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/jointly-learning-to-label-sentences-and","title":"Jointly Learning to Label Sentences and Tokens","arxiv_id":"1811.05949","date":"2018-11-14","proceeding":null,"authors":["Marek Rei","Anders Søgaard"],"abstract":"Learning to construct text representations in end-to-end systems can be\ndifficult, as natural languages are highly compositional and task-specific\nannotated datasets are often limited in size. Methods for directly supervising\nlanguage composition can allow us to guide the models based on existing\nknowledge, regularizing them towards more robust and interpretable\nrepresentations. In this paper, we investigate how objectives at different\ngranularities can be used to learn better language representations and we\npropose an architecture for jointly learning to label sentences and tokens. The\npredictions at each level are combined together using an attention mechanism,\nwith token-level labels also acting as explicit supervision for composing\nsentence-level representations. Our experiments show that by learning to\nperform these tasks jointly on multiple levels, the model achieves substantial\nimprovements for both sentence classification and sequence labeling.","url_abs":"http://arxiv.org/abs/1811.05949v1","url_pdf":"http://arxiv.org/pdf/1811.05949v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"jointly-learning-to-label-sentences-and","repo_url":"https://github.com/MirunaPislar/multi-head-attention-labeller","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"jointly-learning-to-label-sentences-and","repo_url":"https://github.com/marekrei/mltagger","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"grammatical-error-detection","task_name":"Grammatical Error Detection"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-classification","task_name":"Sentence Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grammatical-error-detection-on-conll-2014-a1","task":"Grammatical Error Detection","dataset":"CoNLL-2014 A1","model":"BiLSTM-JOINT (trained on FCE)","rank_in_archive_order":4,"of":8,"metrics":{"F0.5":"22.14"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-detection-on-conll-2014-a2","task":"Grammatical Error Detection","dataset":"CoNLL-2014 A2","model":"BiLSTM-JOINT (trained on FCE)","rank_in_archive_order":5,"of":8,"metrics":{"F0.5":"29.65 "},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-detection-on-fce","task":"Grammatical Error Detection","dataset":"FCE","model":"BiLSTM-JOINT","rank_in_archive_order":2,"of":8,"metrics":{"F0.5":"52.07"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-detection-on-jfleg","task":"Grammatical Error Detection","dataset":"JFLEG","model":"BiLSTM-JOINT (trained on FCE)","rank_in_archive_order":1,"of":1,"metrics":{"F0.5":"52.52"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1811.05949","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}