{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/measuring-and-manipulating-knowledge","title":"Inspecting and Editing Knowledge Representations in Language Models","arxiv_id":"2304.00740","date":"2023-04-03","proceeding":null,"authors":["Evan Hernandez","Belinda Z. Li","Jacob Andreas"],"abstract":"Neural language models (LMs) represent facts about the world described by text. Sometimes these facts derive from training data (in most LMs, a representation of the word \"banana\" encodes the fact that bananas are fruits). Sometimes facts derive from input text itself (a representation of the sentence \"I poured out the bottle\" encodes the fact that the bottle became empty). We describe REMEDI, a method for learning to map statements in natural language to fact encodings in an LM's internal representation system. REMEDI encodings can be used as knowledge editors: when added to LM hidden representations, they modify downstream generation to be consistent with new facts. REMEDI encodings may also be used as probes: when compared to LM representations, they reveal which properties LMs already attribute to mentioned entities, in some cases making it possible to predict when LMs will generate outputs that conflict with background knowledge or input text. REMEDI thus links work on probing, prompting, and LM editing, and offers steps toward general tools for fine-grained inspection and control of knowledge in LMs.","url_abs":"https://arxiv.org/abs/2304.00740v3","url_pdf":"https://arxiv.org/pdf/2304.00740v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"measuring-and-manipulating-knowledge","repo_url":"https://github.com/evandez/remedi","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"anatomy","task_name":"Anatomy"},{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.00740","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.00740"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/evandez/remedi","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"official":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7e57245e2b64561d","entry":"cosine_similarity_float16","repo":"evandez/remedi","repo_kind":"official","path":"remedi/utils/training_utils.py","file_url":"https://github.com/evandez/remedi/blob/HEAD/remedi/utils/training_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e57245e2b64561d"}},{"code_sha256_prefix":"4d22e52c376c83f6","entry":"determine_article","repo":"evandez/remedi","repo_kind":"official","path":"remedi/utils/lang_utils.py","file_url":"https://github.com/evandez/remedi/blob/HEAD/remedi/utils/lang_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d22e52c376c83f6"}},{"code_sha256_prefix":"6a90702d97c87e71","entry":"fixed_split","repo":"evandez/remedi","repo_kind":"official","path":"remedi/utils/training_utils.py","file_url":"https://github.com/evandez/remedi/blob/HEAD/remedi/utils/training_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a90702d97c87e71"}},{"code_sha256_prefix":"a406bf7d756ec808","entry":"random_split","repo":"evandez/remedi","repo_kind":"official","path":"remedi/utils/training_utils.py","file_url":"https://github.com/evandez/remedi/blob/HEAD/remedi/utils/training_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a406bf7d756ec808"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}