{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatically-neutralizing-subjective-bias-in","title":"Automatically Neutralizing Subjective Bias in Text","arxiv_id":"1911.09709","date":"2019-11-21","proceeding":null,"authors":["Reid Pryzant","Richard Diehl Martinez","Nathan Dass","Sadao Kurohashi","Dan Jurafsky","Diyi Yang"],"abstract":"Texts like news, encyclopedias, and some social media strive for objectivity. Yet bias in the form of inappropriate subjectivity - introducing attitudes via framing, presupposing truth, and casting doubt - remains ubiquitous. This kind of bias erodes our collective trust and fuels social conflict. To address this issue, we introduce a novel testbed for natural language generation: automatically bringing inappropriately subjective text into a neutral point of view (\"neutralizing\" biased text). We also offer the first parallel corpus of biased language. The corpus contains 180,000 sentence pairs and originates from Wikipedia edits that removed various framings, presuppositions, and attitudes from biased sentences. Last, we propose two strong encoder-decoder baselines for the task. A straightforward yet opaque CONCURRENT system uses a BERT encoder to identify subjective words as part of the generation process. An interpretable and controllable MODULAR algorithm separates these steps, using (1) a BERT-based classifier to identify problematic words and (2) a novel join embedding through which the classifier can edit the hidden states of the encoder. Large-scale human evaluation across four domains (encyclopedias, news headlines, books, and political speeches) suggests that these algorithms are a first step towards the automatic identification and reduction of bias.","url_abs":"https://arxiv.org/abs/1911.09709v3","url_pdf":"https://arxiv.org/pdf/1911.09709v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatically-neutralizing-subjective-bias-in","repo_url":"https://github.com/rpryzant/neutralizing-bias","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1911.09709","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1911.09709"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rpryzant/neutralizing-bias","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":13},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dc7c732b212737d8","entry":"attempt_load_model","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/models.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc7c732b212737d8"}},{"code_sha256_prefix":"434a3b59116b43ca","entry":"bleu","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/evaluation.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"434a3b59116b43ca"}},{"code_sha256_prefix":"afec3c5fa3707d5d","entry":"bleu_stats","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/evaluation.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afec3c5fa3707d5d"}},{"code_sha256_prefix":"75a5c5c12355b9eb","entry":"build_vocab_maps","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/data.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75a5c5c12355b9eb"}},{"code_sha256_prefix":"bbb7a4c679f05d33","entry":"config_key_string","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/utils.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bbb7a4c679f05d33"}},{"code_sha256_prefix":"2cec995af790c6a4","entry":"config_val_string","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/utils.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cec995af790c6a4"}},{"code_sha256_prefix":"4b9b755d273ab7e3","entry":"detokenize","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"deprecated/eval/score_results.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/deprecated/eval/score_results.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b9b755d273ab7e3"}},{"code_sha256_prefix":"a2d119438207aca4","entry":"extract_attributes","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/data.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a2d119438207aca4"}},{"code_sha256_prefix":"0e5dcca6ddd42a20","entry":"find_matches","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"harvest/gen_data_from_crawl.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/harvest/gen_data_from_crawl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e5dcca6ddd42a20"}},{"code_sha256_prefix":"141219cec6bf7fb1","entry":"get_latest_ckpt","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/models.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"141219cec6bf7fb1"}},{"code_sha256_prefix":"45b1f5e549982076","entry":"get_precisions_recalls_DEPRECIATED","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"baselines/evaluation.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/baselines/evaluation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45b1f5e549982076"}},{"code_sha256_prefix":"c4ae1ec929d0f336","entry":"rm_refs","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"harvest/gen_data_from_crawl.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/harvest/gen_data_from_crawl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4ae1ec929d0f336"}},{"code_sha256_prefix":"30d095bc0d01d94d","entry":"wiki_text_clean","repo":"rpryzant/neutralizing-bias","repo_kind":"official","path":"harvest/crawl_revision_text.py","file_url":"https://github.com/rpryzant/neutralizing-bias/blob/HEAD/harvest/crawl_revision_text.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30d095bc0d01d94d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}