{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bicro-noisy-correspondence-rectification-for","title":"BiCro: Noisy Correspondence Rectification for Multi-modality Data via Bi-directional Cross-modal Similarity Consistency","arxiv_id":"2303.12419","date":"2023-03-22","proceeding":"CVPR 2023 1","authors":["Shuo Yang","Zhaopan Xu","Kai Wang","Yang You","Hongxun Yao","Tongliang Liu","Min Xu"],"abstract":"As one of the most fundamental techniques in multimodal learning, cross-modal matching aims to project various sensory modalities into a shared feature space. To achieve this, massive and correctly aligned data pairs are required for model training. However, unlike unimodal datasets, multimodal datasets are extremely harder to collect and annotate precisely. As an alternative, the co-occurred data pairs (e.g., image-text pairs) collected from the Internet have been widely exploited in the area. Unfortunately, the cheaply collected dataset unavoidably contains many mismatched data pairs, which have been proven to be harmful to the model's performance. To address this, we propose a general framework called BiCro (Bidirectional Cross-modal similarity consistency), which can be easily integrated into existing cross-modal matching models and improve their robustness against noisy data. Specifically, BiCro aims to estimate soft labels for noisy data pairs to reflect their true correspondence degree. The basic idea of BiCro is motivated by that -- taking image-text matching as an example -- similar images should have similar textual descriptions and vice versa. Then the consistency of these two similarities can be recast as the estimated soft labels to train the matching model. The experiments on three popular cross-modal matching datasets demonstrate that our method significantly improves the noise-robustness of various matching models, and surpass the state-of-the-art by a clear margin.","url_abs":"https://arxiv.org/abs/2303.12419v2","url_pdf":"https://arxiv.org/pdf/2303.12419v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bicro-noisy-correspondence-rectification-for","repo_url":"https://github.com/xu5zhao/bicro","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"cross-modal-retrieval-with-noisy","task_name":"Cross-modal retrieval with noisy correspondence"},{"task_slug":"image-text-matching","task_name":"Image-text matching"},{"task_slug":"text-matching","task_name":"Text Matching"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-1","task":"Cross-modal retrieval with noisy correspondence","dataset":"CC152K","model":"BiCro*","rank_in_archive_order":9,"of":15,"metrics":{"Image-to-text R@1":"40.8","Image-to-text R@10":"76.1","Image-to-text R@5":"67.2","R-Sum":"370.2","Text-to-image R@1":"42.1","Text-to-image R@10":"76.4","Text-to-image R@5":"67.6"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-3","task":"Cross-modal retrieval with noisy correspondence","dataset":"COCO-Noisy","model":"BiCro*","rank_in_archive_order":11,"of":17,"metrics":{"Image-to-text R@1":"78.8","Image-to-text R@10":"98.6","Image-to-text R@5":"96.1","R-Sum":"523.2","Text-to-image R@1":"63.7","Text-to-image R@10":"95.7","Text-to-image R@5":"90.3"},"uses_additional_data":false},{"leaderboard":"/sota/cross-modal-retrieval-with-noisy-2","task":"Cross-modal retrieval with noisy correspondence","dataset":"Flickr30K-Noisy","model":"BiCro*","rank_in_archive_order":8,"of":16,"metrics":{"Image-to-text R@1":"78.1","Image-to-text R@10":"97.5","Image-to-text R@5":"94.4","R-Sum":"504.7","Text-to-image R@1":"60.4","Text-to-image R@10":"89.9","Text-to-image R@5":"84.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.12419","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.12419"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xu5zhao/bicro","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/xu5zhao/BiCro","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":11,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"0cd009249fea46c5","entry":"AttentionFiltration","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0cd009249fea46c5"}},{"code_sha256_prefix":"8195ef599b6ffb8c","entry":"ContrastiveLoss","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8195ef599b6ffb8c"}},{"code_sha256_prefix":"bf08ee741b5ed5fb","entry":"EncoderImage","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf08ee741b5ed5fb"}},{"code_sha256_prefix":"4d9bde26decdd78e","entry":"EuclideanDistances","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d9bde26decdd78e"}},{"code_sha256_prefix":"a3a3d96d877b86ac","entry":"GraphReasoning","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3a3d96d877b86ac"}},{"code_sha256_prefix":"65f3e1e2d6b67aff","entry":"SIM_PAIR","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65f3e1e2d6b67aff"}},{"code_sha256_prefix":"e448a20d40c5d40b","entry":"SIM_SELE","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e448a20d40c5d40b"}},{"code_sha256_prefix":"b80dc7ccec06c165","entry":"TextSA","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b80dc7ccec06c165"}},{"code_sha256_prefix":"75ca7806acfbf329","entry":"VisualSA","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75ca7806acfbf329"}},{"code_sha256_prefix":"6939b038c1eaf5b5","entry":"EncoderSimilarity","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6939b038c1eaf5b5"}},{"code_sha256_prefix":"26998b9b97c6ed92","entry":"SGRAF","repo":"xu5zhao/BiCro","repo_kind":"official","path":"model.py","file_url":"https://github.com/xu5zhao/BiCro/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26998b9b97c6ed92"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}