{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-training-with-weak-supervision","title":"Self-Training with Weak Supervision","arxiv_id":"2104.05514","date":"2021-04-12","proceeding":"NAACL 2021 4","authors":["Giannis Karamanolakis","Subhabrata Mukherjee","Guoqing Zheng","Ahmed Hassan Awadallah"],"abstract":"State-of-the-art deep neural networks require large-scale labeled training data that is often expensive to obtain or not available for many tasks. Weak supervision in the form of domain-specific rules has been shown to be useful in such settings to automatically generate weakly labeled training data. However, learning with weak rules is challenging due to their inherent heuristic and noisy nature. An additional challenge is rule coverage and overlap, where prior work on weak supervision only considers instances that are covered by weak rules, thus leaving valuable unlabeled data behind. In this work, we develop a weak supervision framework (ASTRA) that leverages all the available data for a given task. To this end, we leverage task-specific unlabeled data through self-training with a model (student) that considers contextualized representations and predicts pseudo-labels for instances that may not be covered by weak rules. We further develop a rule attention network (teacher) that learns how to aggregate student pseudo-labels with weak rule labels, conditioned on their fidelity and the underlying context of an instance. Finally, we construct a semi-supervised learning objective for end-to-end training with unlabeled data, domain-specific rules, and a small amount of labeled data. Extensive experiments on six benchmark datasets for text classification demonstrate the effectiveness of our approach with significant improvements over state-of-the-art baselines.","url_abs":"https://arxiv.org/abs/2104.05514v1","url_pdf":"https://arxiv.org/pdf/2104.05514v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-training-with-weak-supervision","repo_url":"https://github.com/microsoft/ASTRA","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.05514","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.05514"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/ASTRA","reach":null}],"summary":{"ran":1,"ran_draft_wrong":4,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5bd104cc331427b1","entry":"MajorityVoter","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bd104cc331427b1"}},{"code_sha256_prefix":"f32fd61774a67c5b","entry":"MinEntropyLoss","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f32fd61774a67c5b"}},{"code_sha256_prefix":"8cc45bdab04a9a3b","entry":"create_learning_rate_scheduler","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8cc45bdab04a9a3b"}},{"code_sha256_prefix":"475912483414ae4d","entry":"l1_normalize","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"475912483414ae4d"}},{"code_sha256_prefix":"a9b299e92b4afc95","entry":"to_one_hot","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9b299e92b4afc95"}},{"code_sha256_prefix":"56b51e9592950345","entry":"RAN","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56b51e9592950345"}},{"code_sha256_prefix":"5cf01877f237896d","entry":"construct_rule_network","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cf01877f237896d"}},{"code_sha256_prefix":"c0b2adb20625f132","entry":"normalize_with_random_rule","repo":"microsoft/ASTRA","repo_kind":"official","path":"astra/RuleAttentionNetwork.py","file_url":"https://github.com/microsoft/ASTRA/blob/HEAD/astra/RuleAttentionNetwork.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0b2adb20625f132"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}