{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sae-ssv-supervised-steering-in-sparse","title":"SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models","arxiv_id":"2505.16188","date":"2025-05-22","proceeding":null,"authors":["Zirui He","Mingyu Jin","Bo Shen","Ali Payani","Yongfeng Zhang","Mengnan Du"],"abstract":"Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but controlling their behavior reliably remains challenging, especially in open-ended generation settings. This paper introduces a novel supervised steering approach that operates in sparse, interpretable representation spaces. We employ sparse autoencoders (SAEs)to obtain sparse latent representations that aim to disentangle semantic attributes from model activations. Then we train linear classifiers to identify a small subspace of task-relevant dimensions in latent representations. Finally, we learn supervised steering vectors constrained to this subspace, optimized to align with target behaviors. Experiments across sentiment, truthfulness, and politics polarity steering tasks with multiple LLMs demonstrate that our supervised steering vectors achieve higher success rates with minimal degradation in generation quality compared to existing methods. Further analysis reveals that a notably small subspace is sufficient for effective steering, enabling more targeted and interpretable interventions.","url_abs":"https://arxiv.org/abs/2505.16188v1","url_pdf":"https://arxiv.org/pdf/2505.16188v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"natural-language-understanding","task_name":"Natural Language Understanding"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.16188","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.16188"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Ineedanamehere/SAE-SSV","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2},"by_repo_kind":{"found_in_text":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8e0593ad972bd301","entry":"SSVTrainer","repo":"Ineedanamehere/SAE-SSV","repo_kind":"found_in_text","path":"SAE-SSV_v2/SAESTEER/trainer.py","file_url":"https://github.com/Ineedanamehere/SAE-SSV/blob/HEAD/SAE-SSV_v2/SAESTEER/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e0593ad972bd301"}},{"code_sha256_prefix":"25f3b80038d8caa1","entry":"_CompiledWithFallback","repo":"Ineedanamehere/SAE-SSV","repo_kind":"found_in_text","path":"SAE-SSV_v2/SAESTEER/trainer.py","file_url":"https://github.com/Ineedanamehere/SAE-SSV/blob/HEAD/SAE-SSV_v2/SAESTEER/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"25f3b80038d8caa1"}},{"code_sha256_prefix":"db3a9894f19d4433","entry":"compile_enabled","repo":"Ineedanamehere/SAE-SSV","repo_kind":"found_in_text","path":"SAE-SSV_v2/SAESTEER/trainer.py","file_url":"https://github.com/Ineedanamehere/SAE-SSV/blob/HEAD/SAE-SSV_v2/SAESTEER/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db3a9894f19d4433"}},{"code_sha256_prefix":"2d0609c151404f6d","entry":"maybe_compile","repo":"Ineedanamehere/SAE-SSV","repo_kind":"found_in_text","path":"SAE-SSV_v2/SAESTEER/trainer.py","file_url":"https://github.com/Ineedanamehere/SAE-SSV/blob/HEAD/SAE-SSV_v2/SAESTEER/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d0609c151404f6d"}},{"code_sha256_prefix":"be837b4ee909d273","entry":"resolve_precision_dtype","repo":"Ineedanamehere/SAE-SSV","repo_kind":"found_in_text","path":"SAE-SSV_v2/SAESTEER/trainer.py","file_url":"https://github.com/Ineedanamehere/SAE-SSV/blob/HEAD/SAE-SSV_v2/SAESTEER/trainer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"be837b4ee909d273"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}