{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/preventing-language-models-from-hiding-their","title":"Preventing Language Models From Hiding Their Reasoning","arxiv_id":"2310.18512","date":"2023-10-27","proceeding":null,"authors":["Fabien Roger","Ryan Greenblatt"],"abstract":"Large language models (LLMs) often benefit from intermediate steps of reasoning to generate answers to complex problems. When these intermediate steps of reasoning are used to monitor the activity of the model, it is essential that this explicit reasoning is faithful, i.e. that it reflects what the model is actually reasoning about. In this work, we focus on one potential way intermediate steps of reasoning could be unfaithful: encoded reasoning, where an LLM could encode intermediate steps of reasoning in the generated text in a way that is not understandable to human readers. We show that language models can be trained to make use of encoded reasoning to get higher performance without the user understanding the intermediate steps of reasoning. We argue that, as language models get stronger, this behavior becomes more likely to appear naturally. Finally, we describe a methodology that enables the evaluation of defenses against encoded reasoning, and show that, under the right conditions, paraphrasing successfully prevents even the best encoding schemes we built from encoding more than 3 bits of information per KB of text.","url_abs":"https://arxiv.org/abs/2310.18512v2","url_pdf":"https://arxiv.org/pdf/2310.18512v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"preventing-language-models-from-hiding-their","repo_url":"https://github.com/redwoodresearch/text-steganography-benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.18512","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.18512"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/redwoodresearch/text-steganography-benchmark","reach":{"status":"ok"}}],"summary":{"ran":10,"unverified":4},"by_repo_kind":{"official":{"samples":14,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"8a50be2c3fc08c1f","entry":"ber","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/stats.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/stats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a50be2c3fc08c1f"}},{"code_sha256_prefix":"23214e515396f318","entry":"bitwise_capacity","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/stats.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/stats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23214e515396f318"}},{"code_sha256_prefix":"f51c3c0afa477e83","entry":"bootstrapped_value","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/stats.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/stats.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f51c3c0afa477e83"}},{"code_sha256_prefix":"39f566ebc9b12a49","entry":"clean_string","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/attackers/instruction_attackers.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/attackers/instruction_attackers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"39f566ebc9b12a49"}},{"code_sha256_prefix":"decf3900673f1ca9","entry":"fmt_message","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/log.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/log.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"decf3900673f1ca9"}},{"code_sha256_prefix":"27acc43d7a53ce6a","entry":"hf_model","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/hf.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/hf.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27acc43d7a53ce6a"}},{"code_sha256_prefix":"5b79fb567f19b8ea","entry":"maybe_range","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/utils.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b79fb567f19b8ea"}},{"code_sha256_prefix":"3e94e194e2ac96bf","entry":"openai_parse_names","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/llm.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3e94e194e2ac96bf"}},{"code_sha256_prefix":"ff5f1f8ebdeb67d9","entry":"run_batched_verbose","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/utils.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff5f1f8ebdeb67d9"}},{"code_sha256_prefix":"391296d1e53c0d42","entry":"to_float","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/utils.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"391296d1e53c0d42"}},{"code_sha256_prefix":"bc45dbbc4668e86e","entry":"as_sync","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/llm.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bc45dbbc4668e86e"}},{"code_sha256_prefix":"a368e20f9b123132","entry":"get_cache","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/caching.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/caching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a368e20f9b123132"}},{"code_sha256_prefix":"a351254829f7e834","entry":"hf_models","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/hf.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/hf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a351254829f7e834"}},{"code_sha256_prefix":"5790eaf53e0e0f6b","entry":"sentence_embedding_model","repo":"redwoodresearch/text-steganography-benchmark","repo_kind":"official","path":"textsteg/hf.py","file_url":"https://github.com/redwoodresearch/text-steganography-benchmark/blob/HEAD/textsteg/hf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5790eaf53e0e0f6b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}