{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/one-fish-two-fish-but-not-the-whole-sea","title":"One fish, two fish, but not the whole sea: Alignment reduces language models' conceptual diversity","arxiv_id":"2411.04427","date":"2024-11-07","proceeding":null,"authors":["Sonia K. Murthy","Tomer Ullman","Jennifer Hu"],"abstract":"Researchers in social science and psychology have recently proposed using large language models (LLMs) as replacements for humans in behavioral research. In addition to arguments about whether LLMs accurately capture population-level patterns, this has raised questions about whether LLMs capture human-like conceptual diversity. Separately, it is debated whether post-training alignment (RLHF or RLAIF) affects models' internal diversity. Inspired by human studies, we use a new way of measuring the conceptual diversity of synthetically-generated LLM \"populations\" by relating the internal variability of simulated individuals to the population-level variability. We use this approach to evaluate non-aligned and aligned LLMs on two domains with rich human behavioral data. While no model reaches human-like diversity, aligned models generally display less diversity than their instruction fine-tuned counterparts. Our findings highlight potential trade-offs between increasing models' value alignment and decreasing the diversity of their conceptual representations.","url_abs":"https://arxiv.org/abs/2411.04427v2","url_pdf":"https://arxiv.org/pdf/2411.04427v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"one-fish-two-fish-but-not-the-whole-sea","repo_url":"https://github.com/skmur/onefish-twofish","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2411.04427","atlas_url":"https://app.syntology.ai/?focus=2411.04427","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.04427"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/skmur/onefish-twofish","reach":null}],"summary":{"ran_violates":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"73c6edb5d86ad89c","entry":"computeDeltaE","repo":"skmur/onefish-twofish","repo_kind":"official","path":"analysis/process-data.py","file_url":"https://github.com/skmur/onefish-twofish/blob/HEAD/analysis/process-data.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73c6edb5d86ad89c"}},{"code_sha256_prefix":"df457aec1583f4e6","entry":"compute_delta_e","repo":"skmur/onefish-twofish","repo_kind":"official","path":"src/run-experiments.py","file_url":"https://github.com/skmur/onefish-twofish/blob/HEAD/src/run-experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"df457aec1583f4e6"}},{"code_sha256_prefix":"a50db35d03e3fa4d","entry":"process_file","repo":"skmur/onefish-twofish","repo_kind":"official","path":"analysis/process-data.py","file_url":"https://github.com/skmur/onefish-twofish/blob/HEAD/analysis/process-data.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a50db35d03e3fa4d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}