{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-12263","title":"Multimodal Generative Engine Optimization: Rank Manipulation for Vision-Language Model Rankers","arxiv_id":"2601.12263","date":"2026-01-18","proceeding":null,"authors":["Yixuan Du","Chenxiao Yu","Haoyan Xu","Ziyi Wang","Yue Zhao","Xiyang Hu"],"abstract":"Vision-Language Models (VLMs) integrate visual and textual knowledge into unified representations that increasingly underpin modern retrieval and recommendation systems. However, it remains unclear how reliably these models utilize their cross-modal knowledge when ranking multimodal items, and whether their knowledge grounding can be subverted. In this paper, we expose a fundamental vulnerability in how VLMs apply multimodal knowledge for product ranking: through Multimodal Generative Engine Optimization (MGEO), we show that an adversary can manipulate a VLM's ranking decisions by jointly crafting imperceptible image perturbations and fluent textual suffixes that exploit the model's internal cross-modal knowledge coupling. Using an alternating optimization strategy, MGEO targets the deep interactions between visual and linguistic representations within the VLM, achieving rank manipulations that substantially exceed those of unimodal attacks and heuristic baselines powered by strong commercial models. Our findings reveal that surface-level content quality is insufficient for rank promotion; instead, direct alignment with the model's internal knowledge utilization mechanism is required. These results raise important questions on the faithfulness and robustness of knowledge grounding in multimodal foundation models, and motivate future work on defense mechanisms for multimodal retrieval systems. Code is available at: https://github.com/glad-lab/MGEO","url_abs":"https://arxiv.org/abs/2601.12263","url_pdf":"https://arxiv.org/pdf/2601.12263","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.12263","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.12263"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/glad-lab/MGEO","reach":{"status":"ok"}}],"summary":{"unverified":15},"by_repo_kind":{"found_in_text":{"samples":15,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"38945963615f4620","entry":"bleu_loss","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38945963615f4620"}},{"code_sha256_prefix":"33dd6035c1d1f210","entry":"extract_brand","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"clean_product_names.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/clean_product_names.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"33dd6035c1d1f210"}},{"code_sha256_prefix":"5782194a87c75177","entry":"extract_category_from_path","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"clean_product_names.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/clean_product_names.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5782194a87c75177"}},{"code_sha256_prefix":"f251acd017c5ac40","entry":"fluency_loss","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f251acd017c5ac40"}},{"code_sha256_prefix":"a2502f1d9c6c1d90","entry":"generate_adversarial_image_baseline","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"baseline.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/baseline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2502f1d9c6c1d90"}},{"code_sha256_prefix":"cfc913dec684f279","entry":"generate_adversarial_text_baseline","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"baseline.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/baseline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cfc913dec684f279"}},{"code_sha256_prefix":"d35455c8c31a85ad","entry":"get_banned_tokens","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack_autodan.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack_autodan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d35455c8c31a85ad"}},{"code_sha256_prefix":"5ec37768d9f9fde9","entry":"get_clean_names_prompt","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"clean_product_names.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/clean_product_names.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ec37768d9f9fde9"}},{"code_sha256_prefix":"27171d1b04633a6f","entry":"get_embedding_matrix","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack_autodan.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack_autodan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27171d1b04633a6f"}},{"code_sha256_prefix":"19c802b4dd21c8b8","entry":"get_embeddings","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack_autodan.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack_autodan.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19c802b4dd21c8b8"}},{"code_sha256_prefix":"02a7330e94439059","entry":"get_user_query","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack_config.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack_config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02a7330e94439059"}},{"code_sha256_prefix":"4c8c8c4eb9978e7e","entry":"init_prompt","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"process.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4c8c8c4eb9978e7e"}},{"code_sha256_prefix":"e8b5ab5b2a10f6b5","entry":"process_headtail","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"process.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e8b5ab5b2a10f6b5"}},{"code_sha256_prefix":"80e2a549a1ab5ead","entry":"process_text","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"process.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/process.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80e2a549a1ab5ead"}},{"code_sha256_prefix":"6648c4d59ef6b0f4","entry":"soft_forward","repo":"glad-lab/MGEO","repo_kind":"found_in_text","path":"attack.py","file_url":"https://github.com/glad-lab/MGEO/blob/HEAD/attack.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6648c4d59ef6b0f4"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":null,"prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,264 of the 9,581 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9581,"papers_checked":6264},"entries":[],"not_placed":{"boards":0,"rejected_by_independent_check":0,"refused_by_a_rule":0,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}