{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-from-multiple-noisy-partial-labelers","title":"Learning from Multiple Noisy Partial Labelers","arxiv_id":"2106.04530","date":"2021-06-08","proceeding":null,"authors":["Peilin Yu","Tiffany Ding","Stephen H. Bach"],"abstract":"Programmatic weak supervision creates models without hand-labeled training data by combining the outputs of heuristic labelers. Existing frameworks make the restrictive assumption that labelers output a single class label. Enabling users to create partial labelers that output subsets of possible class labels would greatly expand the expressivity of programmatic weak supervision. We introduce this capability by defining a probabilistic generative model that can estimate the underlying accuracies of multiple noisy partial labelers without ground truth labels. We show how to scale up learning, for example learning on 100k examples in one minute, a 300x speed up compared to a naive implementation. We also prove that this class of models is generically identifiable up to label swapping under mild conditions. We evaluate our framework on three text classification and six object classification tasks. On text tasks, adding partial labels increases average accuracy by 8.6 percentage points. On image tasks, we show that partial labels allow us to approach some zero-shot object classification problems with programmatic weak supervision by using class attributes as partial labelers. On these tasks, our framework has accuracy comparable to recent embedding-based zero-shot learning methods, while using only pre-trained attribute detectors.","url_abs":"https://arxiv.org/abs/2106.04530v2","url_pdf":"https://arxiv.org/pdf/2106.04530v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-from-multiple-noisy-partial-labelers","repo_url":"https://github.com/batsresearch/nplm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"learning-from-multiple-noisy-partial-labelers","repo_url":"https://github.com/batsresearch/yu-aistats22-code","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"text-classification","task_name":"Text Classification"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":"text-classification-1","task_name":"text-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.04530","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.04530"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/batsresearch/yu-aistats22-code","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/batsresearch/nplm","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":5,"ran_violates":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1},"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"2e0222f1fd6d4471","entry":"evaluator","repo":"batsresearch/nplm","repo_kind":"official","path":"nplm/plf/exec.py","file_url":"https://github.com/batsresearch/nplm/blob/HEAD/nplm/plf/exec.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"2e0222f1fd6d4471"}},{"code_sha256_prefix":"041afdc897dbb18a","entry":"executor","repo":"batsresearch/nplm","repo_kind":"official","path":"nplm/plf/exec.py","file_url":"https://github.com/batsresearch/nplm/blob/HEAD/nplm/plf/exec.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"041afdc897dbb18a"}},{"code_sha256_prefix":"c9dac2329ec2721b","entry":"extract_inst_label_pair","repo":"batsresearch/yu-aistats22-code","repo_kind":"listed","path":"run_end_model_ag.py","file_url":"https://github.com/batsresearch/yu-aistats22-code/blob/HEAD/run_end_model_ag.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9dac2329ec2721b"}},{"code_sha256_prefix":"48a7c7193f6372d0","entry":"intercect","repo":"batsresearch/nplm","repo_kind":"official","path":"nplm/lm/labelmodel.py","file_url":"https://github.com/batsresearch/nplm/blob/HEAD/nplm/lm/labelmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"48a7c7193f6372d0"}},{"code_sha256_prefix":"6e31f0275ba89ef3","entry":"load_jkl","repo":"batsresearch/yu-aistats22-code","repo_kind":"listed","path":"run_end_model_ag.py","file_url":"https://github.com/batsresearch/yu-aistats22-code/blob/HEAD/run_end_model_ag.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e31f0275ba89ef3"}},{"code_sha256_prefix":"fcc62568f492b055","entry":"proc_inst","repo":"batsresearch/yu-aistats22-code","repo_kind":"listed","path":"run_end_model_ag.py","file_url":"https://github.com/batsresearch/yu-aistats22-code/blob/HEAD/run_end_model_ag.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fcc62568f492b055"}},{"code_sha256_prefix":"841a5403e54db840","entry":"union","repo":"batsresearch/nplm","repo_kind":"official","path":"nplm/lm/labelmodel.py","file_url":"https://github.com/batsresearch/nplm/blob/HEAD/nplm/lm/labelmodel.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"841a5403e54db840"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}