{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/activation-addition-steering-language-models","title":"Steering Language Models With Activation Engineering","arxiv_id":"2308.10248","date":"2023-08-20","proceeding":null,"authors":["Alexander Matt Turner","Lisa Thiergart","Gavin Leech","David Udell","Juan J. Vazquez","Ulisse Mini","Monte MacDiarmid"],"abstract":"Prompt engineering and finetuning aim to maximize language model performance on a given metric (like toxicity reduction). However, these methods do not fully elicit a model's capabilities. To reduce this gap, we introduce activation engineering: the inference-time modification of activations in order to control (or steer) model outputs. Specifically, we introduce the Activation Addition (ActAdd) technique, which contrasts the intermediate activations on prompt pairs (such as \"Love\" versus \"Hate\") to compute a steering vector (Subramani et al. 2022). By tactically adding in e.g. the \"Love\" - \"Hate\" steering vector during the forward pass, we achieve SOTA on negative-to-positive sentiment shift and detoxification using models including LLaMA-3 and OPT. ActAdd yields inference-time control over high-level output properties (like topic and sentiment) while preserving performance on off-target tasks. ActAdd is lightweight: it does not require any machine optimization and works with a single pair of data points, which enables rapid iteration over steering. ActAdd demonstrates the power of activation engineering.","url_abs":"https://arxiv.org/abs/2308.10248v5","url_pdf":"https://arxiv.org/pdf/2308.10248v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"activation-addition-steering-language-models","repo_url":"https://github.com/montemac/activation_additions","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"activation-addition-steering-language-models","repo_url":"https://github.com/ibm/activation-steering","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"prompt-engineering","task_name":"Prompt Engineering"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"discriminative-fine-tuning","method_name":"Discriminative Fine-Tuning"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-2","method_name":"GPT-2"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"opt","method_name":"OPT"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.10248","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.10248"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/montemac/activation_additions","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ibm/activation-steering","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b4f8d628ed769cb0","entry":"add_metric_cols","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/metrics.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4f8d628ed769cb0"}},{"code_sha256_prefix":"e685cc39acaae19b","entry":"fwd_hooks_from_activ_hooks","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/lenses.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/lenses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e685cc39acaae19b"}},{"code_sha256_prefix":"94fe13d1c85c5bce","entry":"get_function_args","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/logging.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/logging.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94fe13d1c85c5bce"}},{"code_sha256_prefix":"497c3c13a71df2f9","entry":"logits_to_probs_numpy","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/logits.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/logits.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"497c3c13a71df2f9"}},{"code_sha256_prefix":"6ba41b5aaa31756c","entry":"renorm_probs","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/logits.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/logits.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6ba41b5aaa31756c"}},{"code_sha256_prefix":"b5bc96fcc678d944","entry":"trajectory_log_probs","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/lenses.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/lenses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b5bc96fcc678d944"}},{"code_sha256_prefix":"45cf3ae9305a773b","entry":"effectiveness","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/logits.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/logits.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45cf3ae9305a773b"}},{"code_sha256_prefix":"489fbebd4aa24963","entry":"preserve_rng_state","repo":"montemac/activation_additions","repo_kind":"official","path":"activation_additions/completion_utils.py","file_url":"https://github.com/montemac/activation_additions/blob/HEAD/activation_additions/completion_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"489fbebd4aa24963"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}