{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-07158","title":"Topic Is Not Agenda: A Citation-Community Audit of Text Embeddings","arxiv_id":"2605.07158","date":"2026-05-08","proceeding":null,"authors":["Junseon Yoo"],"abstract":"Vector search and retrieval-augmented generation (RAG) rest on the assumption that cosine similarity between text embeddings reflects conceptual relatedness. We measure where this assumption breaks. We build an augmented citation graph over 3.58M scientific papers and partition it via Leiden CPM at two granularities: sub-field (L1) and research-agenda (L2, hierarchical inside each L1). Four state-of-the-art embeddings (Gemini, Qwen3-8B, Qwen3-0.6B, SPECTER2) clear the L1 bar reasonably (45-52% top-10 same-rate) but stop working at L2: only 15-21% of top-10 neighbors share the query's research agenda. In absolute terms, 8 of every 10 retrieved papers are off-agenda. The failure is universal across eight scientific domains and all four models; SPECTER2, despite its citation-based contrastive training, is the weakest. As a diagnostic probe, we test whether the same augmented graph also functions as a retrieval signal: a deliberately simple citation-count rerank reaches 57.7% top-1 L2 on top of LLM-expanded Boolean retrieval and 59.6% on top of plain BM25, on 80 curated agenda queries -- about 9 points above the best cosine retriever (Gemini, 50.6%) and 20 points above BM25 alone (39.3%). The probe isolates a slice of the agenda-matching signal the graph carries but the embeddings miss, connecting recent theoretical limits on single-vector retrieval to a concrete failure mode of scientific RAG.","url_abs":"https://arxiv.org/abs/2605.07158","url_pdf":"https://arxiv.org/pdf/2605.07158","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.07158"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/junseon-yoo/topic-not-agenda","reach":null}],"summary":{"ran":5,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":6,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7a330082f3bf1df3","entry":"is_boilerplate","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/eval/full_sweep.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/eval/full_sweep.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a330082f3bf1df3"}},{"code_sha256_prefix":"8f5becf40fa89e11","entry":"jsd_dict","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/eval/lexical_divergence.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/eval/lexical_divergence.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f5becf40fa89e11"}},{"code_sha256_prefix":"f0ae4e135bfc9af9","entry":"last_token_pool","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/embeddings/embed_qwen3.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/embeddings/embed_qwen3.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f0ae4e135bfc9af9"}},{"code_sha256_prefix":"40842e7a5a492d8b","entry":"stats_for_pids","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/eval/compare_methods.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/eval/compare_methods.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"40842e7a5a492d8b"}},{"code_sha256_prefix":"ac9b1a0705cfa5b0","entry":"tokenize","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/eval/lexical_divergence.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/eval/lexical_divergence.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac9b1a0705cfa5b0"}},{"code_sha256_prefix":"8b62cf65148d5275","entry":"process_community","repo":"junseon-yoo/topic-not-agenda","repo_kind":"found_in_text","path":"code/community/hierarchical_leiden.py","file_url":"https://github.com/junseon-yoo/topic-not-agenda/blob/HEAD/code/community/hierarchical_leiden.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b62cf65148d5275"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}