{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sublinear-time-approximation-of-text","title":"Sublinear Time Approximation of Text Similarity Matrices","arxiv_id":"2112.09631","date":"2021-12-17","proceeding":null,"authors":["Archan Ray","Nicholas Monath","Andrew McCallum","Cameron Musco"],"abstract":"We study algorithms for approximating pairwise similarity matrices that arise in natural language processing. Generally, computing a similarity matrix for $n$ data points requires $\\Omega(n^2)$ similarity computations. This quadratic scaling is a significant bottleneck, especially when similarities are computed via expensive functions, e.g., via transformer models. Approximation methods reduce this quadratic complexity, often by using a small subset of exactly computed similarities to approximate the remainder of the complete pairwise similarity matrix. Significant work focuses on the efficient approximation of positive semidefinite (PSD) similarity matrices, which arise e.g., in kernel methods. However, much less is understood about indefinite (non-PSD) similarity matrices, which often arise in NLP. Motivated by the observation that many of these matrices are still somewhat close to PSD, we introduce a generalization of the popular Nystr\\\"{o}m method to the indefinite setting. Our algorithm can be applied to any similarity matrix and runs in sublinear time in the size of the matrix, producing a rank-$s$ approximation with just $O(ns)$ similarity computations. We show that our method, along with a simple variant of CUR decomposition, performs very well in approximating a variety of similarity matrices arising in NLP tasks. We demonstrate high accuracy of the approximated similarity matrices in the downstream tasks of document classification, sentence similarity, and cross-document coreference.","url_abs":"https://arxiv.org/abs/2112.09631v3","url_pdf":"https://arxiv.org/pdf/2112.09631v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sublinear-time-approximation-of-text","repo_url":"https://github.com/archanray/approximate_similarities","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"document-classification","task_name":"Document Classification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-similarity","task_name":"Sentence Similarity"},{"task_slug":"text-similarity","task_name":"text similarity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.09631","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.09631"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/archanray/approximate_similarities","reach":null}],"summary":{"ran_fixture":2,"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"09b7f1622605155e","entry":"nystrom","repo":"archanray/approximate_similarities","repo_kind":"official","path":"matrix_approximations/approximator.py","file_url":"https://github.com/archanray/approximate_similarities/blob/HEAD/matrix_approximations/approximator.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"09b7f1622605155e"}},{"code_sha256_prefix":"c93dfb0726122640","entry":"nystrom_with_eig_estimate","repo":"archanray/approximate_similarities","repo_kind":"official","path":"matrix_approximations/approximator.py","file_url":"https://github.com/archanray/approximate_similarities/blob/HEAD/matrix_approximations/approximator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c93dfb0726122640"}},{"code_sha256_prefix":"1aaa5dc1066f42c5","entry":"ratio_nystrom","repo":"archanray/approximate_similarities","repo_kind":"official","path":"matrix_approximations/approximator.py","file_url":"https://github.com/archanray/approximate_similarities/blob/HEAD/matrix_approximations/approximator.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1aaa5dc1066f42c5"}},{"code_sha256_prefix":"c95ebb19843a0693","entry":"get_pairwise_labels","repo":"archanray/approximate_similarities","repo_kind":"official","path":"cross_doc_cor/train_pairwise_scorer.py","file_url":"https://github.com/archanray/approximate_similarities/blob/HEAD/cross_doc_cor/train_pairwise_scorer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c95ebb19843a0693"}},{"code_sha256_prefix":"a8a4536d5c10fd39","entry":"train_pairwise_classifier","repo":"archanray/approximate_similarities","repo_kind":"official","path":"cross_doc_cor/train_pairwise_scorer.py","file_url":"https://github.com/archanray/approximate_similarities/blob/HEAD/cross_doc_cor/train_pairwise_scorer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8a4536d5c10fd39"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}