{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/disentangled-representation-learning-for-text","title":"Disentangled Representation Learning for Text-Video Retrieval","arxiv_id":"2203.07111","date":"2022-03-14","proceeding":null,"authors":["Qiang Wang","Yanhao Zhang","Yun Zheng","Pan Pan","Xian-Sheng Hua"],"abstract":"Cross-modality interaction is a critical component in Text-Video Retrieval (TVR), yet there has been little examination of how different influencing factors for computing interaction affect performance. This paper first studies the interaction paradigm in depth, where we find that its computation can be split into two terms, the interaction contents at different granularity and the matching function to distinguish pairs with the same semantics. We also observe that the single-vector representation and implicit intensive function substantially hinder the optimization. Based on these findings, we propose a disentangled framework to capture a sequential and hierarchical representation. Firstly, considering the natural sequential structure in both text and video inputs, a Weighted Token-wise Interaction (WTI) module is performed to decouple the content and adaptively exploit the pair-wise correlations. This interaction can form a better disentangled manifold for sequential inputs. Secondly, we introduce a Channel DeCorrelation Regularization (CDCR) to minimize the redundancy between the components of the compared vectors, which facilitate learning a hierarchical representation. We demonstrate the effectiveness of the disentangled representation on various benchmarks, e.g., surpassing CLIP4Clip largely by +2.9%, +3.1%, +7.9%, +2.3%, +2.8% and +6.5% R@1 on the MSR-VTT, MSVD, VATEX, LSMDC, AcitivityNet, and DiDeMo, respectively.","url_abs":"https://arxiv.org/abs/2203.07111v1","url_pdf":"https://arxiv.org/pdf/2203.07111v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"disentangled-representation-learning-for-text","repo_url":"https://github.com/foolwood/DRL","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"disentangled-representation-learning-for-text","repo_url":"https://github.com/towhee-io/towhee","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-didemo","task":"Video Retrieval","dataset":"DiDeMo","model":"DRL","rank_in_archive_order":25,"of":40,"metrics":{"text-to-video Mean Rank":"11.5","text-to-video Median Rank":"2.0","text-to-video R@1":"49.0","text-to-video R@10":"84.5","text-to-video R@5":"76.5","video-to-text Mean Rank":"7.9","video-to-text Median Rank":"2","video-to-text R@1":"49.9","video-to-text R@10":"83.3"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-msr-vtt-1ka","task":"Video Retrieval","dataset":"MSR-VTT-1kA","model":"DRL","rank_in_archive_order":11,"of":63,"metrics":{"text-to-video Mean Rank":"11.4","text-to-video Median Rank":"1","text-to-video R@1":"53.3","text-to-video R@10":"87.6","text-to-video R@5":"80.3","video-to-text Mean Rank":"7.6","video-to-text Median Rank":"1.0","video-to-text R@1":"56.2","video-to-text R@10":"87.4","video-to-text R@5":"79.9"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2203.07111","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2203.07111"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/towhee-io/towhee","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/foolwood/DRL","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":4,"ran":1,"ran_honours":2,"ran_violates":1,"unverified":4},"by_repo_kind":{"official":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0f786c407fb1ee4c","entry":"swish","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/until_module.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/until_module.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":2,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f786c407fb1ee4c"}},{"code_sha256_prefix":"d741eb912b24880a","entry":"all_gather","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/utils/comm.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/utils/comm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d741eb912b24880a"}},{"code_sha256_prefix":"98f385d847636a3e","entry":"basic_clean","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/tokenization_clip.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/tokenization_clip.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98f385d847636a3e"}},{"code_sha256_prefix":"50e1ffed03f484ec","entry":"gelu","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/until_module.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/until_module.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50e1ffed03f484ec"}},{"code_sha256_prefix":"d919ae32e5e4e616","entry":"get_pairs","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/tokenization_clip.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/tokenization_clip.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d919ae32e5e4e616"}},{"code_sha256_prefix":"2a3280757ce5a198","entry":"reduce_dict","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/utils/comm.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/utils/comm.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a3280757ce5a198"}},{"code_sha256_prefix":"e0960afdb0d64aa7","entry":"warmup_linear","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/optimization.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/optimization.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0960afdb0d64aa7"}},{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/tokenization_clip.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/tokenization_clip.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"b6070984a76e9bf3","entry":"dataloader_msrvtt_test","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/dataloaders/data_dataloaders.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/dataloaders/data_dataloaders.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6070984a76e9bf3"}},{"code_sha256_prefix":"780f499e676f8267","entry":"dataloader_msrvtt_train","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/dataloaders/data_dataloaders.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/dataloaders/data_dataloaders.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"780f499e676f8267"}},{"code_sha256_prefix":"59e4e730a9dc5dc6","entry":"warmup_constant","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/optimization.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/optimization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"59e4e730a9dc5dc6"}},{"code_sha256_prefix":"20b3d25ba1b796e5","entry":"warmup_cosine","repo":"foolwood/DRL","repo_kind":"official","path":"tvr/models/optimization.py","file_url":"https://github.com/foolwood/DRL/blob/HEAD/tvr/models/optimization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20b3d25ba1b796e5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}