{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eliciting-latent-knowledge-from-quirky","title":"Eliciting Latent Knowledge from Quirky Language Models","arxiv_id":"2312.01037","date":"2023-12-02","proceeding":null,"authors":["Alex Mallen","Madeline Brumley","Julia Kharchenko","Nora Belrose"],"abstract":"Eliciting Latent Knowledge (ELK) aims to find patterns in a capable neural network's activations that robustly track the true state of the world, especially in hard-to-verify cases where the model's output is untrusted. To further ELK research, we introduce 12 datasets and a corresponding suite of \"quirky\" language models (LMs) that are finetuned to make systematic errors when answering questions if and only if the keyword \"Bob\" is present in the prompt. We find that, especially in middle layers, linear probes usually report an LM's knowledge independently of what the LM outputs, enabling us to elicit the correct answer despite the model's untruthful output. The best probing method (logistic regression on contrast pairs) recovers 89% of the gap in AUROC between truthful and untruthful contexts, and 75% for questions harder than those used to train the probe. We also find that a mechanistic anomaly detection approach can flag untruthful behavior with 0.95 AUROC. Our results show promise for eliciting reliable knowledge from capable but untrusted models, and facilitates future research empirically investigating ELK methods.","url_abs":"https://arxiv.org/abs/2312.01037v4","url_pdf":"https://arxiv.org/pdf/2312.01037v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eliciting-latent-knowledge-from-quirky","repo_url":"https://github.com/eleutherai/elk-generalization","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"math","task_name":"Math"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.01037","atlas_url":"https://app.syntology.ai/?focus=2312.01037","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.01037"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eleutherai/elk-generalization","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1f88973b76c982a1","entry":"H","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/elk/ccs_losses.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/elk/ccs_losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f88973b76c982a1"}},{"code_sha256_prefix":"1299d3784ab1ff46","entry":"assert_type","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/utils.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1299d3784ab1ff46"}},{"code_sha256_prefix":"ada0375a57f92a80","entry":"bootstrap_auroc","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/anomaly/detect_anomaly.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/anomaly/detect_anomaly.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ada0375a57f92a80"}},{"code_sha256_prefix":"5377efd2b2f26e7e","entry":"ccs_squared_loss","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/elk/ccs_losses.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/elk/ccs_losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5377efd2b2f26e7e"}},{"code_sha256_prefix":"c0626aae8fa29524","entry":"get_logodds","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/anomaly/anomaly_experiment.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/anomaly/anomaly_experiment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0626aae8fa29524"}},{"code_sha256_prefix":"9e7337824581dcce","entry":"register","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/elk/ccs_losses.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/elk/ccs_losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9e7337824581dcce"}},{"code_sha256_prefix":"342384bd62257687","entry":"encode_choice","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/utils.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"342384bd62257687"}},{"code_sha256_prefix":"fad9b59f979eb569","entry":"load_quirky_dataset","repo":"eleutherai/elk-generalization","repo_kind":"official","path":"elk_generalization/datasets/loader_utils.py","file_url":"https://github.com/eleutherai/elk-generalization/blob/HEAD/elk_generalization/datasets/loader_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fad9b59f979eb569"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}