{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ts2-net-token-shift-and-selection-transformer","title":"TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval","arxiv_id":"2207.07852","date":"2022-07-16","proceeding":null,"authors":["Yuqi Liu","Pengfei Xiong","Luhui Xu","Shengming Cao","Qin Jin"],"abstract":"Text-Video retrieval is a task of great practical value and has received increasing attention, among which learning spatial-temporal video representation is one of the research hotspots. The video encoders in the state-of-the-art video retrieval models usually directly adopt the pre-trained vision backbones with the network structure fixed, they therefore can not be further improved to produce the fine-grained spatial-temporal video representation. In this paper, we propose Token Shift and Selection Network (TS2-Net), a novel token shift and selection transformer architecture, which dynamically adjusts the token sequence and selects informative tokens in both temporal and spatial dimensions from input video samples. The token shift module temporally shifts the whole token features back-and-forth across adjacent frames, to preserve the complete token representation and capture subtle movements. Then the token selection module selects tokens that contribute most to local spatial semantics. Based on thorough experiments, the proposed TS2-Net achieves state-of-the-art performance on major text-video retrieval benchmarks, including new records on MSRVTT, VATEX, LSMDC, ActivityNet, and DiDeMo.","url_abs":"https://arxiv.org/abs/2207.07852v1","url_pdf":"https://arxiv.org/pdf/2207.07852v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ts2-net-token-shift-and-selection-transformer","repo_url":"https://github.com/yuqi657/ts2_net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"video-retrieval","task_name":"Video Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-retrieval-on-msr-vtt-1ka","task":"Video Retrieval","dataset":"MSR-VTT-1kA","model":"TS2-Net","rank_in_archive_order":9,"of":63,"metrics":{"text-to-video R@1":"54.0","text-to-video R@10":"87.4","text-to-video R@5":"79.3"},"uses_additional_data":false},{"leaderboard":"/sota/video-retrieval-on-vatex","task":"Video Retrieval","dataset":"VATEX","model":"TS2-Net","rank_in_archive_order":10,"of":13,"metrics":{"text-to-video R@1":"59.1","text-to-video R@10":"95.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.07852","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.07852"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yuqi657/ts2_net","reach":null}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"f9464513ec376e49","entry":"PerturbedTopK","repo":"yuqi657/ts2_net","repo_kind":"official","path":"modules/differential_topk.py","file_url":"https://github.com/yuqi657/ts2_net/blob/HEAD/modules/differential_topk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9464513ec376e49"}},{"code_sha256_prefix":"1bc760a4ba29a8ae","entry":"PerturbedTopKFuntion","repo":"yuqi657/ts2_net","repo_kind":"official","path":"modules/differential_topk.py","file_url":"https://github.com/yuqi657/ts2_net/blob/HEAD/modules/differential_topk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1bc760a4ba29a8ae"}},{"code_sha256_prefix":"d31b93573832315e","entry":"PredictorLG","repo":"yuqi657/ts2_net","repo_kind":"official","path":"modules/differential_topk.py","file_url":"https://github.com/yuqi657/ts2_net/blob/HEAD/modules/differential_topk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d31b93573832315e"}},{"code_sha256_prefix":"a86ec746f7b1c8e7","entry":"VisualTokenSelection","repo":"yuqi657/ts2_net","repo_kind":"official","path":"modules/differential_topk.py","file_url":"https://github.com/yuqi657/ts2_net/blob/HEAD/modules/differential_topk.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a86ec746f7b1c8e7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}