{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/text-visual-prompting-for-efficient-2d","title":"Text-Visual Prompting for Efficient 2D Temporal Video Grounding","arxiv_id":"2303.04995","date":"2023-03-09","proceeding":"CVPR 2023 1","authors":["Yimeng Zhang","Xin Chen","Jinghan Jia","Sijia Liu","Ke Ding"],"abstract":"In this paper, we study the problem of temporal video grounding (TVG), which aims to predict the starting/ending time points of moments described by a text sentence within a long untrimmed video. Benefiting from fine-grained 3D visual features, the TVG techniques have achieved remarkable progress in recent years. However, the high complexity of 3D convolutional neural networks (CNNs) makes extracting dense 3D visual features time-consuming, which calls for intensive memory and computing resources. Towards efficient TVG, we propose a novel text-visual prompting (TVP) framework, which incorporates optimized perturbation patterns (that we call 'prompts') into both visual inputs and textual features of a TVG model. In sharp contrast to 3D CNNs, we show that TVP allows us to effectively co-train vision encoder and language encoder in a 2D TVG model and improves the performance of crossmodal feature fusion using only low-complexity sparse 2D visual features. Further, we propose a Temporal-Distance IoU (TDIoU) loss for efficient learning of TVG. Experiments on two benchmark datasets, Charades-STA and ActivityNet Captions datasets, empirically show that the proposed TVP significantly boosts the performance of 2D TVG (e.g., 9.79% improvement on Charades-STA and 30.77% improvement on ActivityNet Captions) and achieves 5x inference acceleration over TVG using 3D visual features. Codes are available at Open.Intel.","url_abs":"https://arxiv.org/abs/2303.04995v3","url_pdf":"https://arxiv.org/pdf/2303.04995v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"text-visual-prompting-for-efficient-2d","repo_url":"https://github.com/intel/TVP","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"video-grounding","task_name":"Video Grounding"},{"task_slug":"visual-prompting","task_name":"Visual Prompting"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.04995","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.04995"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/intel/TVP","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":11},"by_repo_kind":{"official":{"samples":16,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0cf387da46369b1a","entry":"get_padding","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/data_utils.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0cf387da46369b1a"}},{"code_sha256_prefix":"e9242c50098a6ea2","entry":"image_to_tensor","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/data_utils.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9242c50098a6ea2"}},{"code_sha256_prefix":"f828f173e6c2fea2","entry":"mask_batch_text_tokens","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/data_utils.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/data_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f828f173e6c2fea2"}},{"code_sha256_prefix":"060527bb34794b59","entry":"mish","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/transformers.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/transformers.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"060527bb34794b59"}},{"code_sha256_prefix":"b68769dd52ad6c44","entry":"move_to_cuda","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/dataloader.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/dataloader.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b68769dd52ad6c44"}},{"code_sha256_prefix":"17e89eef72e37efe","entry":"conv3x3","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/grid_feat.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/grid_feat.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"17e89eef72e37efe"}},{"code_sha256_prefix":"9397957c09ba96d3","entry":"dis_iou_loss","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/modeling.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9397957c09ba96d3"}},{"code_sha256_prefix":"f31be4ddaa3ec6c2","entry":"get_random_sample_indices","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/modeling.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f31be4ddaa3ec6c2"}},{"code_sha256_prefix":"707f6cd75833c65d","entry":"get_start_end_idx","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/decoder.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"707f6cd75833c65d"}},{"code_sha256_prefix":"b6331823c359e9d8","entry":"get_video_decoding_kwargs","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/dataset_base.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/dataset_base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b6331823c359e9d8"}},{"code_sha256_prefix":"5a4d93d8be97ccd9","entry":"instance_bce_with_logits","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/modeling.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/modeling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a4d93d8be97ccd9"}},{"code_sha256_prefix":"daa2b8808499576a","entry":"load_decompress_img_from_lmdb_value","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/dataset_base.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/dataset_base.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"daa2b8808499576a"}},{"code_sha256_prefix":"0766f48ad6e01b56","entry":"load_tf_weights_in_bert","repo":"intel/TVP","repo_kind":"official","path":"src/modeling/transformers.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/modeling/transformers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0766f48ad6e01b56"}},{"code_sha256_prefix":"3e81145412a40e36","entry":"parse_with_config","repo":"intel/TVP","repo_kind":"official","path":"src/configs/config.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/configs/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3e81145412a40e36"}},{"code_sha256_prefix":"b28b46ef7776c434","entry":"pyav_decode_stream","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/decoder.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b28b46ef7776c434"}},{"code_sha256_prefix":"f307b4793e8204b7","entry":"temporal_sampling","repo":"intel/TVP","repo_kind":"official","path":"src/datasets/decoder.py","file_url":"https://github.com/intel/TVP/blob/HEAD/src/datasets/decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f307b4793e8204b7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}