{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/trap-targeted-random-adversarial-prompt","title":"TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification","arxiv_id":"2402.12991","date":"2024-02-20","proceeding":null,"authors":["Martin Gubri","Dennis Ulmer","Hwaran Lee","Sangdoo Yun","Seong Joon Oh"],"abstract":"Large Language Model (LLM) services and models often come with legal rules on who can use them and how they must use them. Assessing the compliance of the released LLMs is crucial, as these rules protect the interests of the LLM contributor and prevent misuse. In this context, we describe the novel fingerprinting problem of Black-box Identity Verification (BBIV). The goal is to determine whether a third-party application uses a certain LLM through its chat function. We propose a method called Targeted Random Adversarial Prompt (TRAP) that identifies the specific LLM in use. We repurpose adversarial suffixes, originally proposed for jailbreaking, to get a pre-defined answer from the target LLM, while other models give random answers. TRAP detects the target LLMs with over 95% true positive rate at under 0.2% false positive rate even after a single interaction. TRAP remains effective even if the LLM has minor changes that do not significantly alter the original function.","url_abs":"https://arxiv.org/abs/2402.12991v2","url_pdf":"https://arxiv.org/pdf/2402.12991v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"trap-targeted-random-adversarial-prompt","repo_url":"https://github.com/framartin/trap","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"trap-targeted-random-adversarial-prompt","repo_url":"https://github.com/parameterlab/trap","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.12991","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12991"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/parameterlab/trap","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/framartin/trap","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":7,"unverified":6},"by_repo_kind":{"official":{"samples":13,"ran":7,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"58bff4cf03dd8432","entry":"generate","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/compute_results.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/compute_results.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58bff4cf03dd8432"}},{"code_sha256_prefix":"76bb37c1f3570e55","entry":"generate_random_string","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/generate_csv.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/generate_csv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76bb37c1f3570e55"}},{"code_sha256_prefix":"7e9dd2da021b5534","entry":"get_prompt_target","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/generate_csv.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/generate_csv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e9dd2da021b5534"}},{"code_sha256_prefix":"c7dc4896ac741214","entry":"load_prompts","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/baseline_ppl.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/baseline_ppl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7dc4896ac741214"}},{"code_sha256_prefix":"f28fd11ca495123c","entry":"load_suffixes","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/utils.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f28fd11ca495123c"}},{"code_sha256_prefix":"41c6ae7f8d08a2e9","entry":"load_suffixes_csv","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/utils.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41c6ae7f8d08a2e9"}},{"code_sha256_prefix":"cd0073dd38a0ace7","entry":"load_system_prompts","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/utils.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd0073dd38a0ace7"}},{"code_sha256_prefix":"71b177b3a3ed6078","entry":"combine_prompts","repo":"framartin/trap","repo_kind":"official","path":"llm_attacks/api_experiments/evaluate_api_models.py","file_url":"https://github.com/framartin/trap/blob/HEAD/llm_attacks/api_experiments/evaluate_api_models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71b177b3a3ed6078"}},{"code_sha256_prefix":"ef6fb88be74d8cd0","entry":"generate_n_times","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/compute_results_baseline_api.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/compute_results_baseline_api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef6fb88be74d8cd0"}},{"code_sha256_prefix":"645890ba00019a69","entry":"load_prompts","repo":"parameterlab/trap","repo_kind":"official","path":"llm_attacks/api_experiments/evaluate_api_models.py","file_url":"https://github.com/parameterlab/trap/blob/HEAD/llm_attacks/api_experiments/evaluate_api_models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"645890ba00019a69"}},{"code_sha256_prefix":"d0e94f216f27f804","entry":"make_checking_fn","repo":"framartin/trap","repo_kind":"official","path":"llm_attacks/api_experiments/evaluate_api_models.py","file_url":"https://github.com/framartin/trap/blob/HEAD/llm_attacks/api_experiments/evaluate_api_models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d0e94f216f27f804"}},{"code_sha256_prefix":"93a9f4ea8dfd368b","entry":"openai_sample_once","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/baseline_ppl.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/baseline_ppl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"93a9f4ea8dfd368b"}},{"code_sha256_prefix":"1f4769e15870cc9c","entry":"openai_sample_once","repo":"framartin/trap","repo_kind":"official","path":"detect_llm/compute_results_baseline_api.py","file_url":"https://github.com/framartin/trap/blob/HEAD/detect_llm/compute_results_baseline_api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f4769e15870cc9c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}