{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cos-enhancing-personalization-and-mitigating","title":"CoS: Enhancing Personalization and Mitigating Bias with Context Steering","arxiv_id":"2405.01768","date":"2024-05-02","proceeding":null,"authors":["Jerry Zhi-Yang He","Sashrika Pandey","Mariah L. Schrum","Anca Dragan"],"abstract":"When querying a large language model (LLM), the context, i.e. personal, demographic, and cultural information specific to an end-user, can significantly shape the response of the LLM. For example, asking the model to explain Newton's second law with the context \"I am a toddler\" yields a different answer compared to the context \"I am a physics professor.\" Proper usage of the context enables the LLM to generate personalized responses, whereas inappropriate contextual influence can lead to stereotypical and potentially harmful generations (e.g. associating \"female\" with \"housekeeper\"). In practice, striking the right balance when leveraging context is a nuanced and challenging problem that is often situation-dependent. One common approach to address this challenge is to fine-tune LLMs on contextually appropriate responses. However, this approach is expensive, time-consuming, and not controllable for end-users in different situations. In this work, we propose Context Steering (CoS) - a simple training-free method that can be easily applied to autoregressive LLMs at inference time. By measuring the contextual influence in terms of token prediction likelihood and modulating it, our method enables practitioners to determine the appropriate level of contextual influence based on their specific use case and end-user base. We showcase a variety of applications of CoS including amplifying the contextual influence to achieve better personalization and mitigating unwanted influence for reducing model bias. In addition, we show that we can combine CoS with Bayesian Inference to quantify the extent of hate speech on the internet. We demonstrate the effectiveness of CoS on state-of-the-art LLMs and benchmarks.","url_abs":"https://arxiv.org/abs/2405.01768v1","url_pdf":"https://arxiv.org/pdf/2405.01768v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cos-enhancing-personalization-and-mitigating","repo_url":"https://github.com/sashrikap/context-steering","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-inference","task_name":"Bayesian Inference"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[{"method_slug":"am","method_name":"AM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.01768","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.01768"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sashrikap/context-steering","reach":{"status":"ok"}}],"summary":{"ran":5},"by_repo_kind":{"official":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"19dcd6d1508bc69e","entry":"apply_cos","repo":"sashrikap/context-steering","repo_kind":"official","path":"cos/core.py","file_url":"https://github.com/sashrikap/context-steering/blob/HEAD/cos/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19dcd6d1508bc69e"}},{"code_sha256_prefix":"ba57f243ce1fa1e6","entry":"apply_multi_cos","repo":"sashrikap/context-steering","repo_kind":"official","path":"cos/core.py","file_url":"https://github.com/sashrikap/context-steering/blob/HEAD/cos/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba57f243ce1fa1e6"}},{"code_sha256_prefix":"365eaa8a7a609508","entry":"get_context_pair_dialogs","repo":"sashrikap/context-steering","repo_kind":"official","path":"cos/utils.py","file_url":"https://github.com/sashrikap/context-steering/blob/HEAD/cos/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"365eaa8a7a609508"}},{"code_sha256_prefix":"8619c1b4800a823c","entry":"get_context_pair_texts","repo":"sashrikap/context-steering","repo_kind":"official","path":"cos/utils.py","file_url":"https://github.com/sashrikap/context-steering/blob/HEAD/cos/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8619c1b4800a823c"}},{"code_sha256_prefix":"9db3e56343b66f23","entry":"get_seqs_logits","repo":"sashrikap/context-steering","repo_kind":"official","path":"cos/core.py","file_url":"https://github.com/sashrikap/context-steering/blob/HEAD/cos/core.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9db3e56343b66f23"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}