{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/holistic-features-are-almost-sufficient-for","title":"Holistic Features are almost Sufficient for Text-to-Video Retrieval","arxiv_id":null,"date":"2024-01-01","proceeding":"CVPR 2024 1","authors":["Kaibin Tian","Ruixiang Zhao","Zijie Xin","Bangxiang Lan","Xirong Li"],"abstract":"    For text-to-video retrieval (T2VR) which aims to retrieve unlabeled videos by ad-hoc textual queries CLIP-based methods currently lead the way. Compared to CLIP4Clip which is efficient and compact state-of-the-art models tend to compute video-text similarity through fine-grained cross-modal feature interaction and matching putting their scalability for large-scale T2VR applications into doubt. We propose TeachCLIP enabling a CLIP4Clip based student network to learn from more advanced yet computationally intensive models. In order to create a learning channel to convey fine-grained cross-modal knowledge from a heavy model to the student we add to CLIP4Clip a simple Attentional frame-Feature Aggregation (AFA) block which by design adds no extra storage / computation overhead at the retrieval stage. Frame-text relevance scores calculated by the teacher network are used as soft labels to supervise the attentive weights produced by AFA. Extensive experiments on multiple public datasets justify the viability of the proposed method. TeachCLIP has the same efficiency and compactness as CLIP4Clip yet has near-SOTA effectiveness.    ","url_abs":"http://openaccess.thecvf.com//content/CVPR2024/html/Tian_Holistic_Features_are_almost_Sufficient_for_Text-to-Video_Retrieval_CVPR_2024_paper.html","url_pdf":"http://openaccess.thecvf.com//content/CVPR2024/papers/Tian_Holistic_Features_are_almost_Sufficient_for_Text-to-Video_Retrieval_CVPR_2024_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"holistic-features-are-almost-sufficient-for","repo_url":"https://github.com/ruc-aimc-lab/teachclip","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"text-to-video-retrieval","task_name":"Text to Video Retrieval"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"},{"task_slug":"text-similarity","task_name":"text similarity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-msr-vtt-1ka","task":"Video Retrieval","dataset":"MSR-VTT-1kA","model":"TeachCLIP (ViT-B/16)","rank_in_archive_order":27,"of":63,"metrics":{"text-to-video R@1":"48.0","text-to-video R@10":"83.5","text-to-video R@5":"75.9"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-msr-vtt-1ka","task":"Video Retrieval","dataset":"MSR-VTT-1kA","model":"TeachCLIP","rank_in_archive_order":31,"of":63,"metrics":{"text-to-video R@1":"46.8","text-to-video R@10":"82.6","text-to-video R@5":"74.3"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-vatex","task":"Video Retrieval","dataset":"VATEX","model":"TeachCLIP","rank_in_archive_order":9,"of":13,"metrics":{"text-to-video R@1":"63.6","text-to-video R@10":"96.1","text-to-video R@5":"91.9"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}