Papers › MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrieval

MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrieval

20 Aug 2024arXiv:2408.10575archive 2025-07-28

Haoran Tang, Meng Cao, Jinfa Huang, Ruyang Liu, Peng Jin, Ge Li, Xiaodan Liang

Text-Video Retrieval (TVR) aims to align and associate relevant video content with corresponding natural language queries. Most existing TVR methods are based on large-scale pre-trained vision-language models (e.g., CLIP). However, due to the inherent plain structure of CLIP, few TVR methods explore the multi-scale representations which offer richer contextual information for a more thorough understanding. To this end, we propose MUSE, a multi-scale mamba with linear computational complexity for efficient cross-resolution modeling. Specifically, the multi-scale representations are generated by applying a feature pyramid on the last single-scale feature map. Then, we employ the Mamba structure as an efficient multi-scale learner to jointly learn scale-wise representations. Furthermore, we conduct comprehensive studies to investigate different model structures and designs. Extensive results on three popular benchmarks have validated the superiority of MUSE.

PaperPDFCode

Code

hrtang22/MUSE officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

MambaNatural Language QueriesRetrievalVideo Retrieval

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

ALIGNCLIPMamba

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections