{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/relevance-based-margin-for-contrastively","title":"Relevance-based Margin for Contrastively-trained Video Retrieval Models","arxiv_id":"2204.13001","date":"2022-04-27","proceeding":null,"authors":["Alex Falcon","Swathikiran Sudhakaran","Giuseppe Serra","Sergio Escalera","Oswald Lanz"],"abstract":"Video retrieval using natural language queries has attracted increasing interest due to its relevance in real-world applications, from intelligent access in private media galleries to web-scale video search. Learning the cross-similarity of video and text in a joint embedding space is the dominant approach. To do so, a contrastive loss is usually employed because it organizes the embedding space by putting similar items close and dissimilar items far. This framework leads to competitive recall rates, as they solely focus on the rank of the groundtruth items. Yet, assessing the quality of the ranking list is of utmost importance when considering intelligent retrieval systems, since multiple items may share similar semantics, hence a high relevance. Moreover, the aforementioned framework uses a fixed margin to separate similar and dissimilar items, treating all non-groundtruth items as equally irrelevant. In this paper we propose to use a variable margin: we argue that varying the margin used during training based on how much relevant an item is to a given query, i.e. a relevance-based margin, easily improves the quality of the ranking lists measured through nDCG and mAP. We demonstrate the advantages of our technique using different models on EPIC-Kitchens-100 and YouCook2. We show that even if we carefully tuned the fixed margin, our technique (which does not have the margin as a hyper-parameter) would still achieve better performance. Finally, extensive ablation studies and qualitative analysis support the robustness of our approach. Code will be released at \\url{https://github.com/aranciokov/RelevanceMargin-ICMR22}.","url_abs":"https://arxiv.org/abs/2204.13001v1","url_pdf":"https://arxiv.org/pdf/2204.13001v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"relevance-based-margin-for-contrastively","repo_url":"https://github.com/aranciokov/relevancemargin-icmr22","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"multi-instance-retrieval","task_name":"Multi-Instance Retrieval"},{"task_slug":"natural-language-queries","task_name":"Natural Language Queries"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2204.13001","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2204.13001"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aranciokov/relevancemargin-icmr22","reach":null}],"summary":{"ran_honours":3,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9ae403a06155097d","entry":"calculate_mAP","repo":"aranciokov/relevancemargin-icmr22","repo_kind":"official","path":"src/evaluation/mAP.py","file_url":"https://github.com/aranciokov/relevancemargin-icmr22/blob/HEAD/src/evaluation/mAP.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9ae403a06155097d"}},{"code_sha256_prefix":"6c009f73fd0d3f9e","entry":"get_relevance_verb","repo":"aranciokov/relevancemargin-icmr22","repo_kind":"official","path":"src/train/train_jpose_tripletRelBased.py","file_url":"https://github.com/aranciokov/relevancemargin-icmr22/blob/HEAD/src/train/train_jpose_tripletRelBased.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c009f73fd0d3f9e"}},{"code_sha256_prefix":"7173bd40cab0c090","entry":"get_relevances","repo":"aranciokov/relevancemargin-icmr22","repo_kind":"official","path":"src/train/train_jpose_tripletRelBased.py","file_url":"https://github.com/aranciokov/relevancemargin-icmr22/blob/HEAD/src/train/train_jpose_tripletRelBased.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7173bd40cab0c090"}},{"code_sha256_prefix":"5c8ddea96a126303","entry":"relevance","repo":"aranciokov/relevancemargin-icmr22","repo_kind":"official","path":"src/train/train_jpose_tripletRelBased.py","file_url":"https://github.com/aranciokov/relevancemargin-icmr22/blob/HEAD/src/train/train_jpose_tripletRelBased.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c8ddea96a126303"}},{"code_sha256_prefix":"3ae2bb378626017d","entry":"train_epoch","repo":"aranciokov/relevancemargin-icmr22","repo_kind":"official","path":"src/train/train_mmen_tripletRelBased.py","file_url":"https://github.com/aranciokov/relevancemargin-icmr22/blob/HEAD/src/train/train_mmen_tripletRelBased.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ae2bb378626017d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}