{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contextual-similarity-aggregation-with-self","title":"Contextual Similarity Aggregation with Self-attention for Visual Re-ranking","arxiv_id":"2110.13430","date":"2021-10-26","proceeding":"NeurIPS 2021 12","authors":["Jianbo Ouyang","Hui Wu","Min Wang","Wengang Zhou","Houqiang Li"],"abstract":"In content-based image retrieval, the first-round retrieval result by simple visual feature comparison may be unsatisfactory, which can be refined by visual re-ranking techniques. In image retrieval, it is observed that the contextual similarity among the top-ranked images is an important clue to distinguish the semantic relevance. Inspired by this observation, in this paper, we propose a visual re-ranking method by contextual similarity aggregation with self-attention. In our approach, for each image in the top-K ranking list, we represent it into an affinity feature vector by comparing it with a set of anchor images. Then, the affinity features of the top-K images are refined by aggregating the contextual information with a transformer encoder. Finally, the affinity features are used to recalculate the similarity scores between the query and the top-K images for re-ranking of the latter. To further improve the robustness of our re-ranking model and enhance the performance of our method, a new data augmentation scheme is designed. Since our re-ranking model is not directly involved with the visual feature used in the initial retrieval, it is ready to be applied to retrieval result lists obtained from various retrieval algorithms. We conduct comprehensive experiments on four benchmark datasets to demonstrate the generality and effectiveness of our proposed visual re-ranking method.","url_abs":"https://arxiv.org/abs/2110.13430v1","url_pdf":"https://arxiv.org/pdf/2110.13430v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contextual-similarity-aggregation-with-self","repo_url":"https://github.com/mcc-wh/csa","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"content-based-image-retrieval","task_name":"Content-Based Image Retrieval"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"re-ranking","task_name":"Re-Ranking"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2110.13430","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.13430"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mcc-wh/csa","reach":null}],"summary":{"ran":8,"ran_fixture":1},"by_repo_kind":{"official":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6edc8b341a07610c","entry":"Attention","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6edc8b341a07610c"}},{"code_sha256_prefix":"267b248e3abc0616","entry":"Block","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"267b248e3abc0616"}},{"code_sha256_prefix":"8b3435b073f323ec","entry":"DropPath","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b3435b073f323ec"}},{"code_sha256_prefix":"765b2f83902528ba","entry":"Feature_align","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"765b2f83902528ba"}},{"code_sha256_prefix":"4c2d16dd418b3f2d","entry":"GELU","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c2d16dd418b3f2d"}},{"code_sha256_prefix":"f343584e73054bcf","entry":"Identity","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f343584e73054bcf"}},{"code_sha256_prefix":"c64c51e855184f53","entry":"Mlp","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c64c51e855184f53"}},{"code_sha256_prefix":"0d91672c276a1a7a","entry":"RerankTransformer","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d91672c276a1a7a"}},{"code_sha256_prefix":"3c83cad222232ac6","entry":"drop_path","repo":"mcc-wh/csa","repo_kind":"official","path":"network/RerankTransformer.py","file_url":"https://github.com/mcc-wh/csa/blob/HEAD/network/RerankTransformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c83cad222232ac6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}