{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/internvideo-general-video-foundation-models","title":"InternVideo: General Video Foundation Models via Generative and Discriminative Learning","arxiv_id":"2212.03191","date":"2022-12-06","proceeding":null,"authors":["Yi Wang","Kunchang Li","Yizhuo Li","Yinan He","Bingkun Huang","Zhiyu Zhao","Hongjie Zhang","Jilan Xu","Yi Liu","Zun Wang","Sen Xing","Guo Chen","Junting Pan","Jiashuo Yu","Yali Wang","LiMin Wang","Yu Qiao"],"abstract":"The foundation models have recently shown excellent performance on a variety of downstream tasks in computer vision. However, most existing vision foundation models simply focus on image-level pretraining and adpation, which are limited for dynamic and complex video-level understanding tasks. To fill the gap, we present general video foundation models, InternVideo, by taking advantage of both generative and discriminative self-supervised video learning. Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications. Without bells and whistles, InternVideo achieves state-of-the-art performance on 39 video datasets from extensive tasks including video action recognition/detection, video-language alignment, and open-world video applications. Especially, our methods can obtain 91.1% and 77.2% top-1 accuracy on the challenging Kinetics-400 and Something-Something V2 benchmarks, respectively. All of these results effectively show the generality of our InternVideo for video understanding. The code will be released at https://github.com/OpenGVLab/InternVideo .","url_abs":"https://arxiv.org/abs/2212.03191v2","url_pdf":"https://arxiv.org/pdf/2212.03191v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"internvideo-general-video-foundation-models","repo_url":"https://github.com/opengvlab/internvideo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"internvideo-general-video-foundation-models","repo_url":"https://github.com/yingsen1/unimd","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"action-classification","task_name":"Action Classification"},{"task_slug":"action-recognition-in-videos","task_name":"Action Recognition"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"open-set-action-recognition","task_name":"Open Set Action Recognition"},{"task_slug":"spatio-temporal-action-localization","task_name":"Spatio-Temporal Action Localization"},{"task_slug":"action-recognition","task_name":"Temporal Action Localization"},{"task_slug":"video-question-answering","task_name":"Video Question Answering"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"},{"task_slug":"video-understanding","task_name":"Video Understanding"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"},{"task_slug":"zeroshot-video-question-answer","task_name":"Zero-Shot Video Question Answer"},{"task_slug":"zero-shot-video-retrieval","task_name":"Zero-Shot Video Retrieval"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"internvideo","method_name":"InternVideo"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/action-classification-on-kinetics-400","task":"Action Classification","dataset":"Kinetics-400","model":"InternVideo","rank_in_archive_order":5,"of":207,"metrics":{"Acc@1":"91.1"},"uses_additional_data":false},{"leaderboard":"/sota/action-classification-on-kinetics-600","task":"Action Classification","dataset":"Kinetics-600","model":"InternVideo-T","rank_in_archive_order":5,"of":65,"metrics":{"Top-1 Accuracy":"91.3"},"uses_additional_data":true},{"leaderboard":"/sota/action-classification-on-kinetics-700","task":"Action Classification","dataset":"Kinetics-700","model":"InternVideo-T","rank_in_archive_order":3,"of":36,"metrics":{"Top-1 Accuracy":"84.0"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-on-ava-v2-2","task":"Action Recognition","dataset":"AVA v2.2","model":"InternVideo","rank_in_archive_order":6,"of":38,"metrics":{"mAP":"41.01"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something-1","task":"Action Recognition","dataset":"Something-Something V1","model":"InternVideo","rank_in_archive_order":1,"of":74,"metrics":{"Top 1 Accuracy":"70.0"},"uses_additional_data":true},{"leaderboard":"/sota/action-recognition-in-videos-on-something","task":"Action Recognition","dataset":"Something-Something V2","model":"InternVideo","rank_in_archive_order":3,"of":123,"metrics":{"Top-1 Accuracy":"77.2"},"uses_additional_data":true},{"leaderboard":"/sota/open-set-action-recognition-on-ucf-hmdb","task":"Open Set Action Recognition","dataset":"UCF-HMDB","model":"InternVideo","rank_in_archive_order":1,"of":1,"metrics":{"AUROC":"85.48"},"uses_additional_data":false},{"leaderboard":"/sota/open-set-action-recognition-on-ucf101-mitv2","task":"Open Set Action Recognition","dataset":"UCF101-MiTv2","model":"InternVideo","rank_in_archive_order":1,"of":1,"metrics":{"AUROC":"91.85"},"uses_additional_data":false},{"leaderboard":"/sota/spatio-temporal-action-localization-on-ava","task":"Spatio-Temporal Action Localization","dataset":"AVA-Kinetics","model":"InternVideo","rank_in_archive_order":3,"of":7,"metrics":{"val mAP":"41.01"},"uses_additional_data":true},{"leaderboard":"/sota/temporal-action-localization-on-activitynet","task":"Temporal Action Localization","dataset":"ActivityNet-1.3","model":"InternVideo","rank_in_archive_order":9,"of":33,"metrics":{"mAP":"39.00"},"uses_additional_data":true},{"leaderboard":"/sota/temporal-action-localization-on-fineaction","task":"Temporal Action Localization","dataset":"FineAction","model":"InternVideo","rank_in_archive_order":6,"of":9,"metrics":{"mAP":"17.57"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-action-localization-on-hacs","task":"Temporal Action Localization","dataset":"HACS","model":"InternVideo","rank_in_archive_order":7,"of":12,"metrics":{"Average-mAP":"41.55"},"uses_additional_data":false},{"leaderboard":"/sota/temporal-action-localization-on-thumos14","task":"Temporal Action Localization","dataset":"THUMOS’14","model":"ActionFormer (InternVideo features)","rank_in_archive_order":5,"of":42,"metrics":{"Avg mAP (0.3:0.7)":"71.58"},"uses_additional_data":true},{"leaderboard":"/sota/video-question-answering-on-situated","task":"Video Question Answering","dataset":"STAR Benchmark","model":"InternVideo","rank_in_archive_order":4,"of":17,"metrics":{"Average Accuracy":"58.7"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-activitynet","task":"Video Retrieval","dataset":"ActivityNet","model":"InternVideo","rank_in_archive_order":8,"of":31,"metrics":{"text-to-video R@1":"62.2","video-to-text R@1":"62.8"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-didemo","task":"Video Retrieval","dataset":"DiDeMo","model":"InternVideo","rank_in_archive_order":10,"of":40,"metrics":{"text-to-video R@1":"57.9","video-to-text R@1":"59.1"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-lsmdc","task":"Video Retrieval","dataset":"LSMDC","model":"InternVideo","rank_in_archive_order":8,"of":38,"metrics":{"text-to-video R@1":"34.0","video-to-text R@1":"34.9"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-msr-vtt","task":"Video Retrieval","dataset":"MSR-VTT","model":"InternVideo","rank_in_archive_order":8,"of":40,"metrics":{"text-to-video R@1":"55.2","video-to-text R@1":"57.9"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-msvd","task":"Video Retrieval","dataset":"MSVD","model":"InternVideo","rank_in_archive_order":3,"of":24,"metrics":{"text-to-video R@1":"58.4","video-to-text R@1":"76.3"},"uses_additional_data":true},{"leaderboard":"/sota/video-retrieval-on-vatex","task":"Video Retrieval","dataset":"VATEX","model":"InternVideo","rank_in_archive_order":6,"of":13,"metrics":{"text-to-video R@1":"71.1","video-to-text R@1":"87.2"},"uses_additional_data":false},{"leaderboard":"/sota/visual-question-answering-on-msrvtt-qa-1","task":"Visual Question Answering (VQA)","dataset":"MSRVTT-QA","model":"InternVideo","rank_in_archive_order":6,"of":34,"metrics":{"Accuracy":"0.471"},"uses_additional_data":true},{"leaderboard":"/sota/visual-question-answering-on-msvd-qa-1","task":"Visual Question Answering (VQA)","dataset":"MSVD-QA","model":"InternVideo","rank_in_archive_order":12,"of":36,"metrics":{"Accuracy":"0.555"},"uses_additional_data":true},{"leaderboard":"/sota/visual-question-answering-on-tgif-qa","task":"Visual Question Answering (VQA)","dataset":"TGIF-QA","model":"InternVideo","rank_in_archive_order":2,"of":2,"metrics":{"Accuracy":"0.722"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-question-answer-on-egoschema-1","task":"Zero-Shot Video Question Answer","dataset":"EgoSchema (fullset)","model":"InternVideo","rank_in_archive_order":25,"of":29,"metrics":{"Accuracy":"32.1"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-question-answer-on-star","task":"Zero-Shot Video Question Answer","dataset":"STAR Benchmark","model":"InternVideo","rank_in_archive_order":4,"of":4,"metrics":{"Accuracy":"41.6"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-question-answer-on-tvqa","task":"Zero-Shot Video Question Answer","dataset":"TVQA","model":"InternVideo  (no speech)","rank_in_archive_order":8,"of":9,"metrics":{"Accuracy":"35.9"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-video-retrieval-on-activitynet","task":"Zero-Shot Video Retrieval","dataset":"ActivityNet","model":"InternVideo","rank_in_archive_order":11,"of":12,"metrics":{"text-to-video R@1":"30.7","video-to-text R@1":"31.4"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-video-retrieval-on-didemo","task":"Zero-Shot Video Retrieval","dataset":"DiDeMo","model":"InternVideo","rank_in_archive_order":16,"of":26,"metrics":{"text-to-video R@1":"31.5","text-to-video R@10":"68.2","text-to-video R@5":"57.6","video-to-text R@1":"33.5","video-to-text R@10":"71.1","video-to-text R@5":"60.3"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-video-retrieval-on-lsmdc","task":"Zero-Shot Video Retrieval","dataset":"LSMDC","model":"InternVideo","rank_in_archive_order":8,"of":16,"metrics":{"text-to-video R@1":"17.6","text-to-video R@10":"40.2","text-to-video R@5":"32.4","video-to-text R@1":"13.2","video-to-text R@10":"34.9","video-to-text R@5":"27.8"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-video-retrieval-on-msr-vtt","task":"Zero-Shot Video Retrieval","dataset":"MSR-VTT","model":"InternVideo","rank_in_archive_order":14,"of":41,"metrics":{"text-to-video R@1":"40.7","video-to-text R@1":"39.6"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-video-retrieval-on-msvd","task":"Zero-Shot Video Retrieval","dataset":"MSVD","model":"InternVideo","rank_in_archive_order":11,"of":14,"metrics":{"text-to-video R@1":"43.4","video-to-text R@1":"67.6"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-video-retrieval-on-vatex","task":"Zero-Shot Video Retrieval","dataset":"VATEX","model":"InternVideo","rank_in_archive_order":5,"of":5,"metrics":{"text-to-video R@1":"49.5","video-to-text R@1":"69.5"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.03191","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.03191"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opengvlab/internvideo","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yingsen1/unimd","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aa81d03f07735ab8","entry":"compute_acc","repo":"opengvlab/internvideo","repo_kind":"official","path":"Data/InternVid/utils/basic_utils.py","file_url":"https://github.com/opengvlab/internvideo/blob/HEAD/Data/InternVid/utils/basic_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa81d03f07735ab8"}},{"code_sha256_prefix":"ad12a494674d23fb","entry":"compute_n_params","repo":"opengvlab/internvideo","repo_kind":"official","path":"Data/InternVid/utils/basic_utils.py","file_url":"https://github.com/opengvlab/internvideo/blob/HEAD/Data/InternVid/utils/basic_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ad12a494674d23fb"}},{"code_sha256_prefix":"2d946250dd2f5a4f","entry":"load_json","repo":"opengvlab/internvideo","repo_kind":"official","path":"Data/InternVid/utils/basic_utils.py","file_url":"https://github.com/opengvlab/internvideo/blob/HEAD/Data/InternVid/utils/basic_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d946250dd2f5a4f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}