{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/taken-out-of-context-on-measuring-situational","title":"Taken out of context: On measuring situational awareness in LLMs","arxiv_id":"2309.00667","date":"2023-09-01","proceeding":null,"authors":["Lukas Berglund","Asa Cooper Stickland","Mikita Balesni","Max Kaufmann","Meg Tong","Tomasz Korbak","Daniel Kokotajlo","Owain Evans"],"abstract":"We aim to better understand the emergence of `situational awareness' in large language models (LLMs). A model is situationally aware if it's aware that it's a model and can recognize whether it's currently in testing or deployment. Today's LLMs are tested for safety and alignment before they are deployed. An LLM could exploit situational awareness to achieve a high score on safety tests, while taking harmful actions after deployment. Situational awareness may emerge unexpectedly as a byproduct of model scaling. One way to better foresee this emergence is to run scaling experiments on abilities necessary for situational awareness. As such an ability, we propose `out-of-context reasoning' (in contrast to in-context learning). We study out-of-context reasoning experimentally. First, we finetune an LLM on a description of a test while providing no examples or demonstrations. At test time, we assess whether the model can pass the test. To our surprise, we find that LLMs succeed on this out-of-context reasoning task. Their success is sensitive to the training setup and only works when we apply data augmentation. For both GPT-3 and LLaMA-1, performance improves with model size. These findings offer a foundation for further empirical study, towards predicting and potentially controlling the emergence of situational awareness in LLMs. Code is available at: https://github.com/AsaCooperStickland/situational-awareness-evals.","url_abs":"https://arxiv.org/abs/2309.00667v1","url_pdf":"https://arxiv.org/pdf/2309.00667v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"taken-out-of-context-on-measuring-situational","repo_url":"https://github.com/asacooperstickland/situational-awareness-evals","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.00667","atlas_url":"https://app.syntology.ai/?focus=2309.00667","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.00667"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/asacooperstickland/situational-awareness-evals","reach":{"status":"ok"}}],"summary":{"ran":9},"by_repo_kind":{"official":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"77c14888f47d320d","entry":"convert_runs_to_df","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/wandb_utils.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/wandb_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"77c14888f47d320d"}},{"code_sha256_prefix":"83c71db4e4075edf","entry":"generate_wandb_substring_filter","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/wandb_utils.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/wandb_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"83c71db4e4075edf"}},{"code_sha256_prefix":"ac3c29af7266bc07","entry":"get_cost_per_1k_tokens","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/models/openai_complete.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/models/openai_complete.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ac3c29af7266bc07"}},{"code_sha256_prefix":"aa42dd70be709dc2","entry":"get_openwebtext_path","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/dataset.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa42dd70be709dc2"}},{"code_sha256_prefix":"e01bde09eb15910f","entry":"load_from_json","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/common.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e01bde09eb15910f"}},{"code_sha256_prefix":"9ebcde58a78649e8","entry":"load_from_jsonl","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/common.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9ebcde58a78649e8"}},{"code_sha256_prefix":"91ff9ce03cabfbc5","entry":"load_from_yaml","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/common.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/common.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"91ff9ce03cabfbc5"}},{"code_sha256_prefix":"e35d51ba8013184f","entry":"log_after_retry","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/models/openai_complete.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/models/openai_complete.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e35d51ba8013184f"}},{"code_sha256_prefix":"52a9f448d8cbd82a","entry":"normalize_answer","repo":"asacooperstickland/situational-awareness-evals","repo_kind":"official","path":"sitaevals/models/common.py","file_url":"https://github.com/asacooperstickland/situational-awareness-evals/blob/HEAD/sitaevals/models/common.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52a9f448d8cbd82a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}