{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-06181","title":"Uncertainty Drives Social Bias Changes in Quantized Large Language Models","arxiv_id":"2602.06181","date":"2026-02-05","proceeding":null,"authors":["Stanley Z. Hua","Sanae Lotfi","Irene Y. Chen"],"abstract":"Post-training quantization reduces the computational cost of large language models but fundamentally alters their social biases in ways that aggregate metrics fail to capture. We present the first large-scale study of 50 quantized models evaluated on PostTrainingBiasBench, a unified benchmark of 13 closed- and open-ended bias datasets. We identify a phenomenon we term quantization-induced masked bias flipping, in which up to 21% of responses flip between biased and unbiased states after quantization, despite showing no change in aggregate bias scores. These flips are strongly driven by model uncertainty, where the responses with high uncertainty are 3-11x more likely to change than the confident ones. Quantization strength amplifies this effect, with 4-bit quantized models exhibiting 4-6x more behavioral changes than 8-bit quantized models. Critically, these changes create asymmetric impacts across demographic groups, where bias can worsen by up to 18.6% for some groups while improving by 14.1% for others, yielding misleadingly neutral aggregate outcomes. Larger models show no consistent robustness advantage, and group-specific shifts vary unpredictably across model families. Our findings demonstrate that compression fundamentally alters bias patterns, requiring crucial post-quantization evaluation and interventions to ensure reliability in practice.","url_abs":"https://arxiv.org/abs/2602.06181","url_pdf":"https://arxiv.org/pdf/2602.06181","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.06181","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.06181"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/stan-hua/PostTrainingBiasBenchmark","reach":null}],"summary":{"unverified":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"0da6bdea92530950","entry":"is_provider_online","repo":"stan-hua/PostTrainingBiasBenchmark","repo_kind":"found_in_text","path":"src/utils/llm_gen_wrapper.py","file_url":"https://github.com/stan-hua/PostTrainingBiasBenchmark/blob/HEAD/src/utils/llm_gen_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0da6bdea92530950"}},{"code_sha256_prefix":"51658330a10c0e19","entry":"load_json","repo":"stan-hua/PostTrainingBiasBenchmark","repo_kind":"found_in_text","path":"src/utils/json_utils.py","file_url":"https://github.com/stan-hua/PostTrainingBiasBenchmark/blob/HEAD/src/utils/json_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51658330a10c0e19"}},{"code_sha256_prefix":"e12db6e03a73b082","entry":"update_nested_dict","repo":"stan-hua/PostTrainingBiasBenchmark","repo_kind":"found_in_text","path":"src/utils/json_utils.py","file_url":"https://github.com/stan-hua/PostTrainingBiasBenchmark/blob/HEAD/src/utils/json_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e12db6e03a73b082"}},{"code_sha256_prefix":"7c104b12f730953d","entry":"update_with_existing_data","repo":"stan-hua/PostTrainingBiasBenchmark","repo_kind":"found_in_text","path":"src/utils/json_utils.py","file_url":"https://github.com/stan-hua/PostTrainingBiasBenchmark/blob/HEAD/src/utils/json_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7c104b12f730953d"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}