{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/representation-engineering-a-top-down","title":"Representation Engineering: A Top-Down Approach to AI Transparency","arxiv_id":"2310.01405","date":"2023-10-02","proceeding":null,"authors":["Andy Zou","Long Phan","Sarah Chen","James Campbell","Phillip Guo","Richard Ren","Alexander Pan","Xuwang Yin","Mantas Mazeika","Ann-Kathrin Dombrowski","Shashwat Goel","Nathaniel Li","Michael J. Byun","Zifan Wang","Alex Mallen","Steven Basart","Sanmi Koyejo","Dawn Song","Matt Fredrikson","J. Zico Kolter","Dan Hendrycks"],"abstract":"In this paper, we identify and characterize the emerging area of representation engineering (RepE), an approach to enhancing the transparency of AI systems that draws on insights from cognitive neuroscience. RepE places population-level representations, rather than neurons or circuits, at the center of analysis, equipping us with novel methods for monitoring and manipulating high-level cognitive phenomena in deep neural networks (DNNs). We provide baselines and an initial analysis of RepE techniques, showing that they offer simple yet effective solutions for improving our understanding and control of large language models. We showcase how these methods can provide traction on a wide range of safety-relevant problems, including honesty, harmlessness, power-seeking, and more, demonstrating the promise of top-down transparency research. We hope that this work catalyzes further exploration of RepE and fosters advancements in the transparency and safety of AI systems.","url_abs":"https://arxiv.org/abs/2310.01405v4","url_pdf":"https://arxiv.org/pdf/2310.01405v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"representation-engineering-a-top-down","repo_url":"https://github.com/andyzoujm/representation-engineering","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"representation-engineering-a-top-down","repo_url":"https://github.com/cma1114/activation_steering","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"representation-engineering-a-top-down","repo_url":"https://github.com/kaiyuhe998/rulearn_idea","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"representation-engineering-a-top-down","repo_url":"https://github.com/steering-vectors/steering-vectors","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"representation-engineering-a-top-down","repo_url":"https://github.com/sunblaze-ucb/political_leaning_RepE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/question-answering-on-truthfulqa","task":"Question Answering","dataset":"TruthfulQA","model":"LLaMA-2-Chat-13B + Representation Control (Contrast Vector)","rank_in_archive_order":4,"of":33,"metrics":{"MC1":"0.54"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-truthfulqa","task":"Question Answering","dataset":"TruthfulQA","model":"LLaMA-2-Chat-7B + Representation Control (Contrast Vector)","rank_in_archive_order":5,"of":33,"metrics":{"MC1":"0.48"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.01405","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01405"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunblaze-ucb/political_leaning_RepE","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaiyuhe998/rulearn_idea","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/andyzoujm/representation-engineering","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cma1114/activation_steering","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/steering-vectors/steering-vectors","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":6},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"acc281fc1d7b77bc","entry":"compute_loss","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"lorra_finetune/src/llama2_lorra.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/lorra_finetune/src/llama2_lorra.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acc281fc1d7b77bc"}},{"code_sha256_prefix":"db75f13051c04007","entry":"get_position_ids","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"lorra_finetune/src/train_val_datasets.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/lorra_finetune/src/train_val_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db75f13051c04007"}},{"code_sha256_prefix":"1899aee66262f84f","entry":"get_truncated_outputs","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"lorra_finetune/src/train_val_datasets.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/lorra_finetune/src/train_val_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1899aee66262f84f"}},{"code_sha256_prefix":"97aa4e637d9f5444","entry":"prepare_inputs","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"lorra_finetune/src/train_val_datasets.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/lorra_finetune/src/train_val_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97aa4e637d9f5444"}},{"code_sha256_prefix":"18886c949c98a5c9","entry":"arc_dataset","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe_eval/tasks/arc.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe_eval/tasks/arc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18886c949c98a5c9"}},{"code_sha256_prefix":"2b4c9fbd7f2a466f","entry":"contrast_greedy_search","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe/rep_control_contrast_vec.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe/rep_control_contrast_vec.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b4c9fbd7f2a466f"}},{"code_sha256_prefix":"2ef192a290c2db9a","entry":"csqa_dataset","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe_eval/tasks/csqa.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe_eval/tasks/csqa.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2ef192a290c2db9a"}},{"code_sha256_prefix":"a953412e8172dfef","entry":"forward_contrast_vector","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe/rep_control_contrast_vec.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe/rep_control_contrast_vec.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a953412e8172dfef"}},{"code_sha256_prefix":"8c6b734d5c949090","entry":"project_onto_direction","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe/rep_readers.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe/rep_readers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c6b734d5c949090"}},{"code_sha256_prefix":"1ceba3fc481567ab","entry":"recenter","repo":"andyzoujm/representation-engineering","repo_kind":"official","path":"repe/rep_readers.py","file_url":"https://github.com/andyzoujm/representation-engineering/blob/HEAD/repe/rep_readers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ceba3fc481567ab"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}