{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/style-specific-neurons-for-steering-llms-in","title":"Style-Specific Neurons for Steering LLMs in Text Style Transfer","arxiv_id":"2410.00593","date":"2024-10-01","proceeding":null,"authors":["Wen Lai","Viktor Hangya","Alexander Fraser"],"abstract":"Text style transfer (TST) aims to modify the style of a text without altering its original meaning. Large language models (LLMs) demonstrate superior performance across multiple tasks, including TST. However, in zero-shot setups, they tend to directly copy a significant portion of the input text to the output without effectively changing its style. To enhance the stylistic variety and fluency of the text, we present sNeuron-TST, a novel approach for steering LLMs using style-specific neurons in TST. Specifically, we identify neurons associated with the source and target styles and deactivate source-style-only neurons to give target-style words a higher probability, aiming to enhance the stylistic diversity of the generated text. However, we find that this deactivation negatively impacts the fluency of the generated text, which we address by proposing an improved contrastive decoding method that accounts for rapid token probability shifts across layers caused by deactivated source-style neurons. Empirical experiments demonstrate the effectiveness of the proposed method on six benchmarks, encompassing formality, toxicity, politics, politeness, authorship, and sentiment.","url_abs":"https://arxiv.org/abs/2410.00593v1","url_pdf":"https://arxiv.org/pdf/2410.00593v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"style-specific-neurons-for-steering-llms-in","repo_url":"https://github.com/wenlai-lavine/sneuron-tst","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"style-transfer","task_name":"Style Transfer"},{"task_slug":"text-style-transfoer","task_name":"Text Style Transfer"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.00593","atlas_url":"https://app.syntology.ai/?focus=2410.00593","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.00593"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/huggingface/transformers","reach":null},{"provenance":"deterministic:regex_extraction","url":"https://github.com/wenlai-lavine/sNeuron-TST","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"5eb4d0e1f153b895","entry":"clean_text","repo":"wenlai-lavine/sNeuron-TST","repo_kind":"official","path":"Evaluation/cls/authorship.py","file_url":"https://github.com/wenlai-lavine/sNeuron-TST/blob/HEAD/Evaluation/cls/authorship.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5eb4d0e1f153b895"}},{"code_sha256_prefix":"a0a3e50bf53b36c5","entry":"load_dataset","repo":"wenlai-lavine/sNeuron-TST","repo_kind":"official","path":"Our/gen_contrast_neurons.py","file_url":"https://github.com/wenlai-lavine/sNeuron-TST/blob/HEAD/Our/gen_contrast_neurons.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a0a3e50bf53b36c5"}},{"code_sha256_prefix":"eec474cc8cda034e","entry":"top_k_top_p_filtering","repo":"wenlai-lavine/sNeuron-TST","repo_kind":"official","path":"Our/utils_new.py","file_url":"https://github.com/wenlai-lavine/sNeuron-TST/blob/HEAD/Our/utils_new.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eec474cc8cda034e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}