{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cross-care-assessing-the-healthcare","title":"Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias","arxiv_id":"2405.05506","date":"2024-05-09","proceeding":null,"authors":["Shan Chen","Jack Gallifant","Mingye Gao","Pedro Moreira","Nikolaj Munch","Ajay Muthukkumar","Arvind Rajan","Jaya Kolluri","Amelia Fiske","Janna Hastings","Hugo Aerts","Brian Anthony","Leo Anthony Celi","William G. La Cava","Danielle S. Bitterman"],"abstract":"Large language models (LLMs) are increasingly essential in processing natural languages, yet their application is frequently compromised by biases and inaccuracies originating in their training data. In this study, we introduce Cross-Care, the first benchmark framework dedicated to assessing biases and real world knowledge in LLMs, specifically focusing on the representation of disease prevalence across diverse demographic groups. We systematically evaluate how demographic biases embedded in pre-training corpora like $ThePile$ influence the outputs of LLMs. We expose and quantify discrepancies by juxtaposing these biases against actual disease prevalences in various U.S. demographic groups. Our results highlight substantial misalignment between LLM representation of disease prevalence and real disease prevalence rates across demographic subgroups, indicating a pronounced risk of bias propagation and a lack of real-world grounding for medical applications of LLMs. Furthermore, we observe that various alignment methods minimally resolve inconsistencies in the models' representation of disease prevalence across different languages. For further exploration and analysis, we make all data and a data visualization tool available at: www.crosscare.net.","url_abs":"https://arxiv.org/abs/2405.05506v2","url_pdf":"https://arxiv.org/pdf/2405.05506v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cross-care-assessing-the-healthcare","repo_url":"https://github.com/shan23chen/cross-care","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-visualization","task_name":"Data Visualization"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.05506","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.05506"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shan23chen/cross-care","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":16,"unverified":4},"by_repo_kind":{"official":{"samples":20,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4d08d3d7dced2ae0","entry":"calculate_disease_by_group","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d08d3d7dced2ae0"}},{"code_sha256_prefix":"9a58c6dbe3abe198","entry":"calculate_log_softmax_batch_non_seq2seq","repo":"shan23chen/cross-care","repo_kind":"official","path":"logits_generate/hf_model.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/logits_generate/hf_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a58c6dbe3abe198"}},{"code_sha256_prefix":"8bc7cb8d8bef7fc5","entry":"co_occurrence_within_window","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8bc7cb8d8bef7fc5"}},{"code_sha256_prefix":"6f934cc1b439e9f8","entry":"create_keyword_pattern","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occ_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occ_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f934cc1b439e9f8"}},{"code_sha256_prefix":"dd0d546bb4bf0215","entry":"create_keyword_pattern","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd0d546bb4bf0215"}},{"code_sha256_prefix":"88ce89805a8ffe04","entry":"create_keyword_pattern","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/jsonl_data_filtering_c4.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/jsonl_data_filtering_c4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"88ce89805a8ffe04"}},{"code_sha256_prefix":"59281530c199bbdd","entry":"extract_total_count_from_message","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_infinigram_document_counts.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_infinigram_document_counts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59281530c199bbdd"}},{"code_sha256_prefix":"1194ae31aea58742","entry":"find_co_occurrences","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occ_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occ_analysis.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1194ae31aea58742"}},{"code_sha256_prefix":"b894e1ed19d48aea","entry":"find_duplicates_in_file","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/check_keywords.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/check_keywords.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b894e1ed19d48aea"}},{"code_sha256_prefix":"f1d7a3870630867d","entry":"generate_case_variations","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_infinigram.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_infinigram.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1d7a3870630867d"}},{"code_sha256_prefix":"293bc1ef76e375ef","entry":"process_file","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/jsonl_data_filtering.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/jsonl_data_filtering.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"293bc1ef76e375ef"}},{"code_sha256_prefix":"fe4d91f2f027fe4e","entry":"process_file","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/jsonl_data_filtering_c4.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/jsonl_data_filtering_c4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe4d91f2f027fe4e"}},{"code_sha256_prefix":"c4a0e7d1de5f3a1c","entry":"remove_latex_commands","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/jsonl_data_filtering.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/jsonl_data_filtering.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4a0e7d1de5f3a1c"}},{"code_sha256_prefix":"a56f744d089c3d45","entry":"remove_latex_commands","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/jsonl_data_filtering_c4.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/jsonl_data_filtering_c4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a56f744d089c3d45"}},{"code_sha256_prefix":"94cf61296c1ba3b8","entry":"search_documents_count","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/disease_document_count_infinigram.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/disease_document_count_infinigram.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94cf61296c1ba3b8"}},{"code_sha256_prefix":"f855d60484b63f8c","entry":"tokenize_for_seq2seq","repo":"shan23chen/cross-care","repo_kind":"official","path":"logits_generate/hf_model.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/logits_generate/hf_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f855d60484b63f8c"}},{"code_sha256_prefix":"23ba4360a5bd3294","entry":"calculate_log_softmax_batch_seq2seq","repo":"shan23chen/cross-care","repo_kind":"official","path":"logits_generate/hf_model.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/logits_generate/hf_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"23ba4360a5bd3294"}},{"code_sha256_prefix":"4ea6258eabae7b12","entry":"count_term","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_infinigram.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_infinigram.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ea6258eabae7b12"}},{"code_sha256_prefix":"8d57c440e1ad8631","entry":"process_texts_in_parallel","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occ_analysis.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occ_analysis.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d57c440e1ad8631"}},{"code_sha256_prefix":"bf0bc2de96cec2ca","entry":"search_documents_for_corpus","repo":"shan23chen/cross-care","repo_kind":"official","path":"co_occurrence_generate/co_occurrence_infinigram_document_counts.py","file_url":"https://github.com/shan23chen/cross-care/blob/HEAD/co_occurrence_generate/co_occurrence_infinigram_document_counts.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf0bc2de96cec2ca"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}