{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-12813","title":"REALISTA: Realistic Latent Adversarial Attacks that Elicit LLM Hallucinations","arxiv_id":"2605.12813","date":"2026-05-12","proceeding":"ICML","authors":["Buyun Liang","Jinqi Luo","Liangzu Peng","Kwan Ho Ryan Chan","Darshan Thaker","Kaleab A. Kinfu","Fengrui Tian","Hamed Hassani","René Vidal"],"abstract":"Large language models (LLMs) achieve strong performance across many tasks but remain vulnerable to hallucinations, making it important to systematically evaluate their reliability under realistic adversarial inputs. We formulate hallucination elicitation as a constrained optimization problem, where the goal is to find semantically coherent adversarial prompts that are equivalent to benign user prompts. Existing attack methods remain limited: discrete prompt-based attacks preserve semantic equivalence and coherence but search only over a limited set of prompt variations, while continuous latent-space attacks explore a richer space but often decode into prompts that are no longer valid rephrasings. To address these limitations, we propose REALISTA, a realistic latent-space attack framework. REALISTA constructs an input-dependent dictionary of valid editing directions, each corresponding to a semantically equivalent and coherent rephrasing, and optimizes continuous combinations of these directions in latent space. This design combines the optimization flexibility of continuous attacks with the semantic realism of discrete rephrasing-based attacks. Experiments demonstrate that REALISTA achieves superior or comparable performance to state-of-the-art realistic attacks on open-source LLMs and, crucially, succeeds in attacking large reasoning models under free-form response settings, where prior realistic attacks fail. Code is available at https://github.com/Buyun-Liang/REALISTA.","url_abs":"https://arxiv.org/abs/2605.12813","url_pdf":"https://arxiv.org/pdf/2605.12813","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.12813","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.12813"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/Buyun-Liang/REALISTA","reach":null}],"summary":{"ran_draft_wrong":12,"ran_violates":1,"ran_fixture":1,"ran_honours":2,"unverified":9},"by_repo_kind":{"found_in_text":{"samples":25,"ran":16,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2f58159205f333ca","entry":"_choice_headers","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f58159205f333ca"}},{"code_sha256_prefix":"c56d7e3b9a495229","entry":"_choice_token_ids","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c56d7e3b9a495229"}},{"code_sha256_prefix":"86e1206f2614ec22","entry":"_prob_cells","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"86e1206f2614ec22"}},{"code_sha256_prefix":"687f1f1da535773b","entry":"_select_best_trial","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"687f1f1da535773b"}},{"code_sha256_prefix":"dd218e17560cd7be","entry":"_select_stage1_initializations","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd218e17560cd7be"}},{"code_sha256_prefix":"66d6ca7c75949643","entry":"feasibility_check","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"66d6ca7c75949643"}},{"code_sha256_prefix":"f78445a8d4feec00","entry":"format_probs","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f78445a8d4feec00"}},{"code_sha256_prefix":"0e6755ce2c27c984","entry":"get_full_input_embeds","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e6755ce2c27c984"}},{"code_sha256_prefix":"33179574bc5bb0fb","entry":"get_probs","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33179574bc5bb0fb"}},{"code_sha256_prefix":"4c2b9df401898003","entry":"get_prompt","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c2b9df401898003"}},{"code_sha256_prefix":"256bbe4baa3a523f","entry":"hallucination_judge_score","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"256bbe4baa3a523f"}},{"code_sha256_prefix":"1ea47c37afad7783","entry":"latent_from_delta","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1ea47c37afad7783"}},{"code_sha256_prefix":"7fbc99fb0076b337","entry":"obj_fun","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7fbc99fb0076b337"}},{"code_sha256_prefix":"dfdbcd1b1d4dcc87","entry":"project_l1_nonneg","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfdbcd1b1d4dcc87"}},{"code_sha256_prefix":"90c470e61ee1a1fd","entry":"projection","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90c470e61ee1a1fd"}},{"code_sha256_prefix":"09e38826e42900b7","entry":"reconstruct_from_latent","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09e38826e42900b7"}},{"code_sha256_prefix":"548d217c9fe4923e","entry":"AttackResultRecorder","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"548d217c9fe4923e"}},{"code_sha256_prefix":"431ed128bf63eb08","entry":"GPT","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"431ed128bf63eb08"}},{"code_sha256_prefix":"54e15db7b9a7752b","entry":"PLD","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54e15db7b9a7752b"}},{"code_sha256_prefix":"14355910d5b8c376","entry":"_print_choice_legend","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14355910d5b8c376"}},{"code_sha256_prefix":"a82b86806665d63e","entry":"_print_final_result","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a82b86806665d63e"}},{"code_sha256_prefix":"ebcfec676c7f1aca","entry":"_print_header","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ebcfec676c7f1aca"}},{"code_sha256_prefix":"794eab336f0599d6","entry":"generate_full_response","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"794eab336f0599d6"}},{"code_sha256_prefix":"cb54a7a753780f37","entry":"obj_fun_with_prompt","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb54a7a753780f37"}},{"code_sha256_prefix":"9b76becfd7ca34c7","entry":"set_seed","repo":"Buyun-Liang/REALISTA","repo_kind":"found_in_text","path":"src/realista.py","file_url":"https://github.com/Buyun-Liang/REALISTA/blob/HEAD/src/realista.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9b76becfd7ca34c7"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}