{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-06588","title":"vLLM Hook v0: A Plug-in for Programming Model Internals on vLLM","arxiv_id":"2603.06588","date":"2026-02-02","proceeding":null,"authors":["Ching-Yun Ko","Pin-Yu Chen"],"abstract":"Modern artificial intelligence (AI) models are deployed on inference engines to optimize runtime efficiency and resource allocation, particularly for transformer-based large language models (LLMs). The vLLM project is a major open-source library to support model serving and inference. However, the current implementation of vLLM limits programmability of the internal states of deployed models. This prevents the use of popular test-time model alignment and enhancement methods. For example, it prevents the detection of adversarial prompts based on attention patterns or the adjustment of model responses based on activation steering. To bridge this critical gap, we present vLLM Hook, an opensource plug-in to enable the programming of internal states for vLLM models. Based on a configuration file specifying which internal states to capture, vLLM Hook provides seamless integration to vLLM and supports two essential features: passive programming and active programming. For passive programming, vLLM Hook probes the selected internal states for subsequent analysis, while keeping the model generation intact. For active programming, vLLM Hook enables efficient intervention of model generation by altering the selected internal states. In addition to presenting the core functions of vLLM Hook, in version 0, we demonstrate 3 use cases including prompt injection detection, enhanced retrieval-augmented retrieval (RAG), and activation steering. Finally, we welcome the community's contribution to improve vLLM Hook via https://github.com/ibm/vllm-hook.","url_abs":"https://arxiv.org/abs/2603.06588","url_pdf":"https://arxiv.org/pdf/2603.06588","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.06588","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.06588"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ibm/vllm-hook","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/IBM/vLLM-Hook","reach":null}],"summary":{"ran_draft_wrong":2,"ran":2,"unverified":10},"by_repo_kind":{"found_in_text":{"samples":14,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8aed59834d398386","entry":"_decompress","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8aed59834d398386"}},{"code_sha256_prefix":"75b3a649b9d5bc37","entry":"_patched_chat_full_generator","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"75b3a649b9d5bc37"}},{"code_sha256_prefix":"f9bdf2359e95aaf4","entry":"_patched_generate","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9bdf2359e95aaf4"}},{"code_sha256_prefix":"59005275a7ff04f3","entry":"_serialize_probes","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"59005275a7ff04f3"}},{"code_sha256_prefix":"74082005041388ef","entry":"_patched_completion_response","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"74082005041388ef"}},{"code_sha256_prefix":"f62fca75d92975a9","entry":"_patched_create_engine_config","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f62fca75d92975a9"}},{"code_sha256_prefix":"041bc7208e5d9330","entry":"_patched_llm_generate","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"041bc7208e5d9330"}},{"code_sha256_prefix":"e1c1d76370fbbdf0","entry":"_trim_probes","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e1c1d76370fbbdf0"}},{"code_sha256_prefix":"633a65a1a9563a0a","entry":"load_and_merge_hs_cache","repo":"ibm/vllm-hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/run_utils.py","file_url":"https://github.com/ibm/vllm-hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/run_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"633a65a1a9563a0a"}},{"code_sha256_prefix":"9bcf375ead197dfe","entry":"load_from_shm","repo":"ibm/vllm-hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/shm_utils.py","file_url":"https://github.com/ibm/vllm-hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/shm_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9bcf375ead197dfe"}},{"code_sha256_prefix":"b2bf8210c576741d","entry":"register","repo":"IBM/vLLM-Hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","file_url":"https://github.com/IBM/vLLM-Hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/_hook_plugin.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b2bf8210c576741d"}},{"code_sha256_prefix":"5b29f86ad1c28979","entry":"setup_shm","repo":"ibm/vllm-hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/shm_utils.py","file_url":"https://github.com/ibm/vllm-hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/shm_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5b29f86ad1c28979"}},{"code_sha256_prefix":"c37dc034d57a12ba","entry":"unpack_hidden_states","repo":"ibm/vllm-hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/run_utils.py","file_url":"https://github.com/ibm/vllm-hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/run_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c37dc034d57a12ba"}},{"code_sha256_prefix":"7f3c22af2eff141a","entry":"unpack_qk","repo":"ibm/vllm-hook","repo_kind":"found_in_text","path":"vllm_hook_plugins/vllm_hook_plugins/run_utils.py","file_url":"https://github.com/ibm/vllm-hook/blob/HEAD/vllm_hook_plugins/vllm_hook_plugins/run_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7f3c22af2eff141a"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}