{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/similarity-search-for-efficient-active","title":"Similarity Search for Efficient Active Learning and Search of Rare Concepts","arxiv_id":"2007.00077","date":"2020-06-30","proceeding":null,"authors":["Cody Coleman","Edward Chou","Julian Katz-Samuels","Sean Culatana","Peter Bailis","Alexander C. Berg","Robert Nowak","Roshan Sumbaly","Matei Zaharia","I. Zeki Yalniz"],"abstract":"Many active learning and search approaches are intractable for large-scale industrial settings with billions of unlabeled examples. Existing approaches search globally for the optimal examples to label, scaling linearly or even quadratically with the unlabeled data. In this paper, we improve the computational efficiency of active learning and search methods by restricting the candidate pool for labeling to the nearest neighbors of the currently labeled set instead of scanning over all of the unlabeled data. We evaluate several selection strategies in this setting on three large-scale computer vision datasets: ImageNet, OpenImages, and a de-identified and aggregated dataset of 10 billion images provided by a large internet company. Our approach achieved similar mean average precision and recall as the traditional global approach while reducing the computational cost of selection by up to three orders of magnitude, thus enabling web-scale active learning.","url_abs":"https://arxiv.org/abs/2007.00077v2","url_pdf":"https://arxiv.org/pdf/2007.00077v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"similarity-search-for-efficient-active","repo_url":"https://github.com/webis-de/small-text","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"},{"task_slug":"computational-efficiency","task_name":"Computational Efficiency"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.00077","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.00077"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/webis-de/small-text","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2cde9d5fa0bd3c31","entry":"argselect","repo":"webis-de/small-text","repo_kind":"listed","path":"small_text/query_strategies/base.py","file_url":"https://github.com/webis-de/small-text/blob/HEAD/small_text/query_strategies/base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cde9d5fa0bd3c31"}},{"code_sha256_prefix":"c5b64298377c59c2","entry":"greedy_coreset","repo":"webis-de/small-text","repo_kind":"listed","path":"small_text/query_strategies/coresets.py","file_url":"https://github.com/webis-de/small-text/blob/HEAD/small_text/query_strategies/coresets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c5b64298377c59c2"}},{"code_sha256_prefix":"179dd0d84e26e346","entry":"lightweight_coreset","repo":"webis-de/small-text","repo_kind":"listed","path":"small_text/query_strategies/coresets.py","file_url":"https://github.com/webis-de/small-text/blob/HEAD/small_text/query_strategies/coresets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"179dd0d84e26e346"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}