{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-2021-image-similarity-dataset-and","title":"The 2021 Image Similarity Dataset and Challenge","arxiv_id":"2106.09672","date":"2021-06-17","proceeding":null,"authors":["Matthijs Douze","Giorgos Tolias","Ed Pizzi","Zoë Papakipos","Lowik Chanussot","Filip Radenovic","Tomas Jenicek","Maxim Maximov","Laura Leal-Taixé","Ismail Elezi","Ondřej Chum","Cristian Canton Ferrer"],"abstract":"This paper introduces a new benchmark for large-scale image similarity detection. This benchmark is used for the Image Similarity Challenge at NeurIPS'21 (ISC2021). The goal is to determine whether a query image is a modified copy of any image in a reference corpus of size 1~million. The benchmark features a variety of image transformations such as automated transformations, hand-crafted image edits and machine-learning based manipulations. This mimics real-life cases appearing in social media, for example for integrity-related problems dealing with misinformation and objectionable content. The strength of the image manipulations, and therefore the difficulty of the benchmark, is calibrated according to the performance of a set of baseline approaches. Both the query and reference set contain a majority of \"distractor\" images that do not match, which corresponds to a real-life needle-in-haystack setting, and the evaluation metric reflects that. We expect the DISC21 benchmark to promote image copy detection as an important and challenging computer vision task and refresh the state of the art. Code and data are available at https://github.com/facebookresearch/isc2021","url_abs":"https://arxiv.org/abs/2106.09672v4","url_pdf":"https://arxiv.org/pdf/2106.09672v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-2021-image-similarity-dataset-and","repo_url":"https://github.com/facebookresearch/isc2021","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"copy-detection","task_name":"Copy Detection"},{"task_slug":"image-similarity-detection","task_name":"Image Similarity Detection"},{"task_slug":"misinformation","task_name":"Misinformation"}],"methods":[],"datasets_introduced":[{"slug":"disc21","name":"DISC21","full_name":"Dataset for ISC 2021"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-similarity-detection-on-disc21-dev","task":"Image Similarity Detection","dataset":"DISC21 dev","model":"HOW+ASMK","rank_in_archive_order":1,"of":4,"metrics":{"Time (ms)":"150","hardware":"Tesla P-100","w/o normalization":"17.32","with normalization":"37.15"},"uses_additional_data":false},{"leaderboard":"/sota/image-similarity-detection-on-disc21-dev","task":"Image Similarity Detection","dataset":"DISC21 dev","model":"Multigrain 1500 dim","rank_in_archive_order":2,"of":4,"metrics":{"Time (ms)":"23","dimension":"1500","hardware":"Tesla V100","w/o normalization":"16.47","with normalization":"36.42"},"uses_additional_data":false},{"leaderboard":"/sota/image-similarity-detection-on-disc21-dev","task":"Image Similarity Detection","dataset":"DISC21 dev","model":"GIST PCA 256","rank_in_archive_order":3,"of":4,"metrics":{"dimension":"256","hardware":"CPU, 2.2 GHz, 40 threads","w/o normalization":"15.56"},"uses_additional_data":false},{"leaderboard":"/sota/image-similarity-detection-on-disc21-dev","task":"Image Similarity Detection","dataset":"DISC21 dev","model":"GIST 960 dim","rank_in_archive_order":4,"of":4,"metrics":{"Time (ms)":"0.55","dimension":"960","hardware":"CPU, 2.2 GHz, 40 threads","w/o normalization":"14.42"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2106.09672","atlas_url":"https://app.syntology.ai/?focus=2106.09672","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.09672"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/facebookresearch/isc2021","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"d463de5ad4a0af63","entry":"gem_npy","repo":"facebookresearch/isc2021","repo_kind":"official","path":"baselines/GeM_baseline.py","file_url":"https://github.com/facebookresearch/isc2021/blob/HEAD/baselines/GeM_baseline.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d463de5ad4a0af63"}},{"code_sha256_prefix":"ba8b8627a8a632a5","entry":"resnet_activation_map","repo":"facebookresearch/isc2021","repo_kind":"official","path":"baselines/GeM_baseline.py","file_url":"https://github.com/facebookresearch/isc2021/blob/HEAD/baselines/GeM_baseline.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ba8b8627a8a632a5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}