{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/zeroi2v-zero-cost-adaptation-of-pre-trained","title":"ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Video","arxiv_id":"2310.01324","date":"2023-10-02","proceeding":null,"authors":["Xinhao Li","Yuhan Zhu","LiMin Wang"],"abstract":"Adapting image models to the video domain has emerged as an efficient paradigm for solving video recognition tasks. Due to the huge number of parameters and effective transferability of image models, performing full fine-tuning is less efficient and even unnecessary. Thus, recent research is shifting its focus toward parameter-efficient image-to-video adaptation. However, these adaptation strategies inevitably introduce extra computational costs to deal with the domain gap and temporal modeling in videos. In this paper, we present a new adaptation paradigm (ZeroI2V) to transfer the image transformers to video recognition tasks (i.e., introduce zero extra cost to the original models during inference). To achieve this goal, we present two core designs. First, to capture the dynamics in videos and reduce the difficulty of image-to-video adaptation, we exploit the flexibility of self-attention and introduce spatial-temporal dual-headed attention (STDHA). This approach efficiently endows the image transformers with temporal modeling capability at zero extra parameters and computation. Second, to handle the domain gap between images and videos, we propose a linear adaption strategy that utilizes lightweight densely placed linear adapters to fully transfer the frozen image models to video recognition. Thanks to the customized linear design, all newly added adapters could be easily merged with the original modules through structural reparameterization after training, enabling zero extra cost during inference. Extensive experiments on representative fully-supervised and few-shot video recognition benchmarks showcase that ZeroI2V can match or even outperform previous state-of-the-art methods while enjoying superior parameter and inference efficiency.","url_abs":"https://arxiv.org/abs/2310.01324v2","url_pdf":"https://arxiv.org/pdf/2310.01324v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"zeroi2v-zero-cost-adaptation-of-pre-trained","repo_url":"https://github.com/mcg-nju/zeroi2v","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"zeroi2v-zero-cost-adaptation-of-pre-trained","repo_url":"https://github.com/leexinhao/ZeroI2V","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"video-recognition","task_name":"Video Recognition"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"ZeroI2V ViT-L/14","rank_in_archive_order":38,"of":207,"metrics":{"Acc@1":"87.2","Acc@5":"97.6"},"uses_additional_data":false},{"leaderboard":"/sota/action-recognition-in-videos-on-hmdb-51","task":"Action Recognition","dataset":"HMDB-51","model":"ZeroI2V ViT-L/14","rank_in_archive_order":11,"of":77,"metrics":{"Average accuracy of 3 splits":"83.4"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"ZeroI2V ViT-L/14","rank_in_archive_order":25,"of":123,"metrics":{"Top-1 Accuracy":"72.2","Top-5 Accuracy":"93.0"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-ucf101","task":"Action Recognition","dataset":"UCF101","model":"ZeroI2V ViT-L/14","rank_in_archive_order":9,"of":91,"metrics":{"3-fold Accuracy":"98.6"},"uses_additional_data":true}],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.01324","atlas_url":"https://app.syntology.ai/?focus=2310.01324","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.01324"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mcg-nju/zeroi2v","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leexinhao/ZeroI2V","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_violates":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"266e5daeaa370734","entry":"get_new_result","repo":"mcg-nju/zeroi2v","repo_kind":"official","path":"tools/weight_reparam.py","file_url":"https://github.com/mcg-nju/zeroi2v/blob/HEAD/tools/weight_reparam.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"266e5daeaa370734"}},{"code_sha256_prefix":"5fdb8261d23f30da","entry":"get_old_result","repo":"mcg-nju/zeroi2v","repo_kind":"official","path":"tools/weight_reparam.py","file_url":"https://github.com/mcg-nju/zeroi2v/blob/HEAD/tools/weight_reparam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5fdb8261d23f30da"}},{"code_sha256_prefix":"5facd6f287d20e06","entry":"reparameterize_LinearAdapter","repo":"mcg-nju/zeroi2v","repo_kind":"official","path":"tools/weight_reparam.py","file_url":"https://github.com/mcg-nju/zeroi2v/blob/HEAD/tools/weight_reparam.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5facd6f287d20e06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}