{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/navigating-the-ocean-of-biases-political-bias","title":"Exploring the Jungle of Bias: Political Bias Attribution in Language Models via Dependency Analysis","arxiv_id":"2311.08605","date":"2023-11-15","proceeding":null,"authors":["David F. Jenny","Yann Billeter","Mrinmaya Sachan","Bernhard Schölkopf","Zhijing Jin"],"abstract":"The rapid advancement of Large Language Models (LLMs) has sparked intense debate regarding the prevalence of bias in these models and its mitigation. Yet, as exemplified by both results on debiasing methods in the literature and reports of alignment-related defects from the wider community, bias remains a poorly understood topic despite its practical relevance. To enhance the understanding of the internal causes of bias, we analyse LLM bias through the lens of causal fairness analysis, which enables us to both comprehend the origins of bias and reason about its downstream consequences and mitigation. To operationalize this framework, we propose a prompt-based method for the extraction of confounding and mediating attributes which contribute to the LLM decision process. By applying Activity Dependency Networks (ADNs), we then analyse how these attributes influence an LLM's decision process. We apply our method to LLM ratings of argument quality in political debates. We find that the observed disparate treatment can at least in part be attributed to confounding and mitigating attributes and model misalignment, and discuss the consequences of our findings for human-AI alignment and bias mitigation. Our code and data are at https://github.com/david-jenny/LLM-Political-Study.","url_abs":"https://arxiv.org/abs/2311.08605v2","url_pdf":"https://arxiv.org/pdf/2311.08605v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"navigating-the-ocean-of-biases-political-bias","repo_url":"https://github.com/david-jenny/llm-political-study","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"fairness","task_name":"Fairness"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2311.08605","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.08605"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/david-jenny/llm-political-study","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":11,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5bffff3cb2fb4b88","entry":"compute_distance_metrics","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/network_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/network_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bffff3cb2fb4b88"}},{"code_sha256_prefix":"9a9e3ce0a7cf2f30","entry":"compute_node_activity_and_pcpn","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/network_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/network_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a9e3ce0a7cf2f30"}},{"code_sha256_prefix":"572b4d90507e8119","entry":"compute_partial_correlations","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/network_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/network_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"572b4d90507e8119"}},{"code_sha256_prefix":"b72f26a8ac91386c","entry":"create_speaker_df","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/plotter_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/plotter_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b72f26a8ac91386c"}},{"code_sha256_prefix":"9b2f0a8e0c9cc1a4","entry":"gen_bootstrap_data","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/statistical_test_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/statistical_test_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b2f0a8e0c9cc1a4"}},{"code_sha256_prefix":"1edc977e616f8ecc","entry":"gen_subset_data","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/statistical_test_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/statistical_test_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1edc977e616f8ecc"}},{"code_sha256_prefix":"651ae257712c7c1b","entry":"get_slice_contextual_variables","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/llm_measurements/query_llm.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/llm_measurements/query_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"651ae257712c7c1b"}},{"code_sha256_prefix":"a8ea4c06a5fba55d","entry":"get_slice_metadata","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/llm_measurements/query_llm.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/llm_measurements/query_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8ea4c06a5fba55d"}},{"code_sha256_prefix":"99a2a61c0258d6cc","entry":"get_speaker_metadata","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/llm_measurements/query_llm.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/llm_measurements/query_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99a2a61c0258d6cc"}},{"code_sha256_prefix":"2bc5a8a09a4696f8","entry":"std","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/statistical_test_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/statistical_test_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2bc5a8a09a4696f8"}},{"code_sha256_prefix":"309d6c16f18bf747","entry":"transform_str_categories_to_flags","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/plotter_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/plotter_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"309d6c16f18bf747"}},{"code_sha256_prefix":"51fcdc53b4801135","entry":"detokenizer","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/cpd_debates/cpd_debate_scraper.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/cpd_debates/cpd_debate_scraper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51fcdc53b4801135"}},{"code_sha256_prefix":"e50b8f5c380e0ce4","entry":"encoding_getter","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/cpd_debates/cpd_debate_scraper.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/cpd_debates/cpd_debate_scraper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e50b8f5c380e0ce4"}},{"code_sha256_prefix":"6496089f2dd2deb1","entry":"load_slice_and_measured_observables","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"utils/plotter_utils.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/utils/plotter_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6496089f2dd2deb1"}},{"code_sha256_prefix":"957d76c0f71ca73e","entry":"tokenizer","repo":"david-jenny/llm-political-study","repo_kind":"official","path":"datasets/cpd_debates/cpd_debate_scraper.py","file_url":"https://github.com/david-jenny/llm-political-study/blob/HEAD/datasets/cpd_debates/cpd_debate_scraper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"957d76c0f71ca73e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}