{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fair-classification-with-partial-feedback-an","title":"Fair Classification with Partial Feedback: An Exploration-Based Data Collection Approach","arxiv_id":"2402.11338","date":"2024-02-17","proceeding":null,"authors":["Vijay Keswani","Anay Mehrotra","L. Elisa Celis"],"abstract":"In many predictive contexts (e.g., credit lending), true outcomes are only observed for samples that were positively classified in the past. These past observations, in turn, form training datasets for classifiers that make future predictions. However, such training datasets lack information about the outcomes of samples that were (incorrectly) negatively classified in the past and can lead to erroneous classifiers. We present an approach that trains a classifier using available data and comes with a family of exploration strategies to collect outcome data about subpopulations that otherwise would have been ignored. For any exploration strategy, the approach comes with guarantees that (1) all sub-populations are explored, (2) the fraction of false positives is bounded, and (3) the trained classifier converges to a ``desired'' classifier. The right exploration strategy is context-dependent; it can be chosen to improve learning guarantees and encode context-specific group fairness properties. Evaluation on real-world datasets shows that this approach consistently boosts the quality of collected outcome data and improves the fraction of true positives for all groups, with only a small reduction in predictive utility.","url_abs":"https://arxiv.org/abs/2402.11338v2","url_pdf":"https://arxiv.org/pdf/2402.11338v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fair-classification-with-partial-feedback-an","repo_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.11338","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.11338"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback","reach":null}],"summary":{"ran_violates":3,"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"3d139c5361a10aad","entry":"clf_cons_predict","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3d139c5361a10aad"}},{"code_sha256_prefix":"7ee3dbb16ee4b826","entry":"clf_cons_predict_prob","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7ee3dbb16ee4b826"}},{"code_sha256_prefix":"923798b9c40292d2","entry":"get_constrained_classifier","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"923798b9c40292d2"}},{"code_sha256_prefix":"0d4afb32e84215a0","entry":"get_fair_constrained_classifier","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d4afb32e84215a0"}},{"code_sha256_prefix":"41f72d6f03130ad3","entry":"get_rev","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"41f72d6f03130ad3"}},{"code_sha256_prefix":"549a2707b6d65eb6","entry":"sigmoid","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"549a2707b6d65eb6"}},{"code_sha256_prefix":"fef8b9fd9fb75796","entry":"Algorithm","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fef8b9fd9fb75796"}},{"code_sha256_prefix":"f151017f4370277a","entry":"Exp_Exp_Algorithm","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f151017f4370277a"}},{"code_sha256_prefix":"2a4709db9662962c","entry":"get_rev_max_classifier","repo":"vijaykeswani/fair-classification-with-partial-feedback","repo_kind":"official","path":"algorithms.py","file_url":"https://github.com/vijaykeswani/fair-classification-with-partial-feedback/blob/HEAD/algorithms.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a4709db9662962c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}