{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simple-but-effective-techniques-to-reduce","title":"End-to-End Bias Mitigation by Modelling Biases in Corpora","arxiv_id":"1909.06321","date":"2019-09-13","proceeding":"ACL 2020 6","authors":["Rabeeh Karimi Mahabadi","Yonatan Belinkov","James Henderson"],"abstract":"Several recent studies have shown that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models that fail to generalize to out-of-domain datasets and are likely to perform poorly in real-world scenarios. We propose two learning strategies to train neural models, which are more robust to such biases and transfer better to out-of-domain datasets. The biases are specified in terms of one or more bias-only models, which learn to leverage the dataset biases. During training, the bias-only models' predictions are used to adjust the loss of the base model to reduce its reliance on biases by down-weighting the biased examples and focusing the training on the hard examples. We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data. Results show that our debiasing methods greatly improve robustness in all settings and better transfer to other textual entailment datasets. Our code and data are publicly available in \\url{https://github.com/rabeehk/robust-nli}.","url_abs":"https://arxiv.org/abs/1909.06321v3","url_pdf":"https://arxiv.org/pdf/1909.06321v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simple-but-effective-techniques-to-reduce","repo_url":"https://github.com/rabeehk/robust-nli","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"simple-but-effective-techniques-to-reduce","repo_url":"https://github.com/rabeehk/robust-nli-fixed","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"fact-verification","task_name":"Fact Verification"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1909.06321","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.06321"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rabeehk/robust-nli","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rabeehk/robust-nli-fixed","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran_honours":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"87c72ef874700e2d","entry":"convert_2d_prob_to_3d","repo":"rabeehk/robust-nli","repo_kind":"official","path":"src/losses.py","file_url":"https://github.com/rabeehk/robust-nli/blob/HEAD/src/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87c72ef874700e2d"}},{"code_sha256_prefix":"b1951f4f25d8e36c","entry":"log","repo":"rabeehk/robust-nli","repo_kind":"official","path":"src/losses.py","file_url":"https://github.com/rabeehk/robust-nli/blob/HEAD/src/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1951f4f25d8e36c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}