{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tradeoffs-between-alignment-and-helpfulness","title":"Tradeoffs Between Alignment and Helpfulness in Language Models with Representation Engineering","arxiv_id":"2401.16332","date":"2024-01-29","proceeding":null,"authors":["Yotam Wolf","Noam Wies","Dorin Shteyman","Binyamin Rothberg","Yoav Levine","Amnon Shashua"],"abstract":"Language model alignment has become an important component of AI safety, allowing safe interactions between humans and language models, by enhancing desired behaviors and inhibiting undesired ones. It is often done by tuning the model or inserting preset aligning prompts. Recently, representation engineering, a method which alters the model's behavior via changing its representations post-training, was shown to be effective in aligning LLMs (Zou et al., 2023a). Representation engineering yields gains in alignment oriented tasks such as resistance to adversarial attacks and reduction of social biases, but was also shown to cause a decrease in the ability of the model to perform basic tasks. In this paper we study the tradeoff between the increase in alignment and decrease in helpfulness of the model. We propose a theoretical framework which provides bounds for these two quantities, and demonstrate their relevance empirically. First, we find that under the conditions of our framework, alignment can be guaranteed with representation engineering, and at the same time that helpfulness is harmed in the process. Second, we show that helpfulness is harmed quadratically with the norm of the representation engineering vector, while the alignment increases linearly with it, indicating a regime in which it is efficient to use representation engineering. We validate our findings empirically, and chart the boundaries to the usefulness of representation engineering for alignment.","url_abs":"https://arxiv.org/abs/2401.16332v4","url_pdf":"https://arxiv.org/pdf/2401.16332v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tradeoffs-between-alignment-and-helpfulness","repo_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.16332","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.16332"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"b300584883c76483","entry":"indices_of_k_most_probable_tokens","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"generate_plots_code/separate_by_dr_visualization.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/generate_plots_code/separate_by_dr_visualization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b300584883c76483"}},{"code_sha256_prefix":"3a9b9c5f5c4584c2","entry":"initialize_result_containers","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"harmfulness_experiments/harmfulness_helpfulness.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/harmfulness_experiments/harmfulness_helpfulness.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3a9b9c5f5c4584c2"}},{"code_sha256_prefix":"78ae5c277264f51c","entry":"k_most_common_indices","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"generate_plots_code/separate_by_dr_visualization.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/generate_plots_code/separate_by_dr_visualization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"78ae5c277264f51c"}},{"code_sha256_prefix":"59e79b011501cec4","entry":"keys_for_values","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"generate_plots_code/separate_by_dr_visualization.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/generate_plots_code/separate_by_dr_visualization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59e79b011501cec4"}},{"code_sha256_prefix":"cb6c9ff66d5194bb","entry":"project_onto_direction","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"repe/rep_readers.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/repe/rep_readers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb6c9ff66d5194bb"}},{"code_sha256_prefix":"f7f3b221cb4cdae8","entry":"recenter","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"repe/rep_readers.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/repe/rep_readers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f7f3b221cb4cdae8"}},{"code_sha256_prefix":"33d4b87d6c62162b","entry":"setup_model_and_tokenizer","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"harmfulness_experiments/harmfulness_helpfulness.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/harmfulness_experiments/harmfulness_helpfulness.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"33d4b87d6c62162b"}},{"code_sha256_prefix":"835f0561c14cf899","entry":"setup_model_and_tokenizer","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"utils/generate_code_with_REPE.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/utils/generate_code_with_REPE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"835f0561c14cf899"}},{"code_sha256_prefix":"f7053de92acde5b9","entry":"setup_model_and_tokenizer","repo":"dorin133/repe_alignment_helpfulness_tradeoff","repo_kind":"official","path":"fairness_experiments/fairness_safety.py","file_url":"https://github.com/dorin133/repe_alignment_helpfulness_tradeoff/blob/HEAD/fairness_experiments/fairness_safety.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f7053de92acde5b9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}