{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/an-investigation-of-critical-issues-in-bias","title":"Are Bias Mitigation Techniques for Deep Learning Effective?","arxiv_id":"2104.00170","date":"2021-04-01","proceeding":null,"authors":["Robik Shrestha","Kushal Kafle","Christopher Kanan"],"abstract":"A critical problem in deep learning is that systems learn inappropriate biases, resulting in their inability to perform well on minority groups. This has led to the creation of multiple algorithms that endeavor to mitigate bias. However, it is not clear how effective these methods are. This is because study protocols differ among papers, systems are tested on datasets that fail to test many forms of bias, and systems have access to hidden knowledge or are tuned specifically to the test set. To address this, we introduce an improved evaluation protocol, sensible metrics, and a new dataset, which enables us to ask and answer critical questions about bias mitigation algorithms. We evaluate seven state-of-the-art algorithms using the same network architecture and hyperparameter selection policy across three benchmark datasets. We introduce a new dataset called Biased MNIST that enables assessment of robustness to multiple bias sources. We use Biased MNIST and a visual question answering (VQA) benchmark to assess robustness to hidden biases. Rather than only tuning to the test set distribution, we study robustness across different tuning distributions, which is critical because for many applications the test distribution may not be known during development. We find that algorithms exploit hidden biases, are unable to scale to multiple forms of bias, and are highly sensitive to the choice of tuning set. Based on our findings, we implore the community to adopt more rigorous assessment of future bias mitigation methods. All data, code, and results are publicly available at: https://github.com/erobic/bias-mitigators.","url_abs":"https://arxiv.org/abs/2104.00170v4","url_pdf":"https://arxiv.org/pdf/2104.00170v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"an-investigation-of-critical-issues-in-bias","repo_url":"https://github.com/erobic/bias-mitigators","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2104.00170","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.00170"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/erobic/bias-mitigators","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"unverified":9},"by_repo_kind":{"official":{"samples":11,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d9def42110729a85","entry":"conv1x1","repo":"erobic/bias-mitigators","repo_kind":"official","path":"models/variable_width_resnet.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/models/variable_width_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9def42110729a85"}},{"code_sha256_prefix":"160bb14bd76201b4","entry":"conv3x3","repo":"erobic/bias-mitigators","repo_kind":"official","path":"models/variable_width_resnet.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/models/variable_width_resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"160bb14bd76201b4"}},{"code_sha256_prefix":"1287501b7e22e4fd","entry":"build_balanced_loader","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/dataloader_factory.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/dataloader_factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1287501b7e22e4fd"}},{"code_sha256_prefix":"1bc2c477c675cc91","entry":"build_bias_retriever","repo":"erobic/bias-mitigators","repo_kind":"official","path":"utils/bias_retrievers.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/utils/bias_retrievers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bc2c477c675cc91"}},{"code_sha256_prefix":"55b5e002e8f5ed32","entry":"build_model","repo":"erobic/bias-mitigators","repo_kind":"official","path":"models/model_factory.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/models/model_factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55b5e002e8f5ed32"}},{"code_sha256_prefix":"98046f924da50fa6","entry":"create_biased_mnist_datasets","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/biased_mnist_dataset.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/biased_mnist_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"98046f924da50fa6"}},{"code_sha256_prefix":"a6b383abeeed187d","entry":"create_celebA_dataset","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/celebA_dataset.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/celebA_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6b383abeeed187d"}},{"code_sha256_prefix":"7108c190526863b8","entry":"create_celebA_datasets","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/celebA_dataset.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/celebA_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7108c190526863b8"}},{"code_sha256_prefix":"5adcdbe43eb9ff88","entry":"dataset_to_xy","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/biased_mnist_dataset.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/biased_mnist_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5adcdbe43eb9ff88"}},{"code_sha256_prefix":"d9175f7ca12deae4","entry":"get_transform_celebA","repo":"erobic/bias-mitigators","repo_kind":"official","path":"datasets/celebA_dataset.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/datasets/celebA_dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9175f7ca12deae4"}},{"code_sha256_prefix":"bed7fe4dae1cc07a","entry":"resnet10vw","repo":"erobic/bias-mitigators","repo_kind":"official","path":"models/variable_width_resnet.py","file_url":"https://github.com/erobic/bias-mitigators/blob/HEAD/models/variable_width_resnet.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bed7fe4dae1cc07a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}