{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-hierarchical-multi-task-approach-for","title":"A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks","arxiv_id":"1811.06031","date":"2018-11-14","proceeding":null,"authors":["Victor Sanh","Thomas Wolf","Sebastian Ruder"],"abstract":"Much effort has been devoted to evaluate whether multi-task learning can be\nleveraged to learn rich representations that can be used in various Natural\nLanguage Processing (NLP) down-stream applications. However, there is still a\nlack of understanding of the settings in which multi-task learning has a\nsignificant effect. In this work, we introduce a hierarchical model trained in\na multi-task learning setup on a set of carefully selected semantic tasks. The\nmodel is trained in a hierarchical fashion to introduce an inductive bias by\nsupervising a set of low level tasks at the bottom layers of the model and more\ncomplex tasks at the top layers of the model. This model achieves\nstate-of-the-art results on a number of tasks, namely Named Entity Recognition,\nEntity Mention Detection and Relation Extraction without hand-engineered\nfeatures or external NLP tools like syntactic parsers. The hierarchical\ntraining supervision induces a set of shared semantic representations at lower\nlayers of the model. We show that as we move from the bottom to the top layers\nof the model, the hidden states of the layers tend to represent more complex\nsemantic information.","url_abs":"http://arxiv.org/abs/1811.06031v2","url_pdf":"http://arxiv.org/pdf/1811.06031v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-hierarchical-multi-task-approach-for","repo_url":"https://github.com/huggingface/hmtl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"relation-extraction","task_name":"Relation Extraction"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/relation-extraction-on-ace-2005","task":"Relation Extraction","dataset":"ACE 2005","model":"Hierarchical Multi-task","rank_in_archive_order":11,"of":30,"metrics":{"Cross Sentence":"No","NER Micro F1":"87.5","RE Micro F1":"62.7","Sentence Encoder":"ELMo"},"uses_additional_data":true}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1811.06031","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1811.06031"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/hmtl","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"13e0a60292dfc23c","entry":"canonicalize_clusters","repo":"huggingface/hmtl","repo_kind":"listed","path":"hmtl/dataset_readers/coref_ace.py","file_url":"https://github.com/huggingface/hmtl/blob/HEAD/hmtl/dataset_readers/coref_ace.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13e0a60292dfc23c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}