{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spmtrack-spatio-temporal-parameter-efficient","title":"SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking","arxiv_id":"2503.18338","date":"2025-03-24","proceeding":"CVPR 2025 1","authors":["Wenrui Cai","Qingjie Liu","Yunhong Wang"],"abstract":"Most state-of-the-art trackers adopt one-stream paradigm, using a single Vision Transformer for joint feature extraction and relation modeling of template and search region images. However, relation modeling between different image patches exhibits significant variations. For instance, background regions dominated by target-irrelevant information require reduced attention allocation, while foreground, particularly boundary areas, need to be be emphasized. A single model may not effectively handle all kinds of relation modeling simultaneously. In this paper, we propose a novel tracker called SPMTrack based on mixture-of-experts tailored for visual tracking task (TMoE), combining the capability of multiple experts to handle diverse relation modeling more flexibly. Benefiting from TMoE, we extend relation modeling from image pairs to spatio-temporal context, further improving tracking accuracy with minimal increase in model parameters. Moreover, we employ TMoE as a parameter-efficient fine-tuning method, substantially reducing trainable parameters, which enables us to train SPMTrack of varying scales efficiently and preserve the generalization ability of pretrained models to achieve superior performance. We conduct experiments on seven datasets, and experimental results demonstrate that our method significantly outperforms current state-of-the-art trackers. The source code is available at https://github.com/WenRuiCai/SPMTrack.","url_abs":"https://arxiv.org/abs/2503.18338v1","url_pdf":"https://arxiv.org/pdf/2503.18338v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spmtrack-spatio-temporal-parameter-efficient","repo_url":"https://github.com/wenruicai/spmtrack","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":null,"task_name":"Relation"},{"task_slug":"visual-object-tracking","task_name":"Visual Object Tracking"},{"task_slug":"visual-tracking","task_name":"Visual Tracking"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"adopt","method_name":"ADOPT"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/visual-object-tracking-on-got-10k","task":"Visual Object Tracking","dataset":"GOT-10k","model":"SPMTrack-G","rank_in_archive_order":3,"of":42,"metrics":{"Average Overlap":"81","Success Rate 0.5":"89.2","Success Rate 0.75":"82.3"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-got-10k","task":"Visual Object Tracking","dataset":"GOT-10k","model":"SPMTrack-L","rank_in_archive_order":5,"of":42,"metrics":{"Average Overlap":"80","Success Rate 0.5":"89.4","Success Rate 0.75":"79.9"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-got-10k","task":"Visual Object Tracking","dataset":"GOT-10k","model":"SPMTrack-B","rank_in_archive_order":17,"of":42,"metrics":{"Average Overlap":"76.5","Success Rate 0.5":"85.9","Success Rate 0.75":"76.3"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot","task":"Visual Object Tracking","dataset":"LaSOT","model":"SPMTrack-G","rank_in_archive_order":1,"of":46,"metrics":{"AUC":"77.4","Normalized Precision":"86.6","Precision":"85"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot","task":"Visual Object Tracking","dataset":"LaSOT","model":"SPMTrack-L","rank_in_archive_order":2,"of":46,"metrics":{"AUC":"76.8","Normalized Precision":"85.9","Precision":"84"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-lasot","task":"Visual Object Tracking","dataset":"LaSOT","model":"SPMTrack-B","rank_in_archive_order":8,"of":46,"metrics":{"AUC":"74.9","Normalized Precision":"84","Precision":"81.7"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-needforspeed","task":"Visual Object Tracking","dataset":"NeedForSpeed","model":"SPMTrack-B","rank_in_archive_order":7,"of":10,"metrics":{"AUC":"0.674"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-otb-2015","task":"Visual Object Tracking","dataset":"OTB-2015","model":"SPMTrack-B","rank_in_archive_order":1,"of":18,"metrics":{"AUC":"0.727"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-tnl2k","task":"Visual Object Tracking","dataset":"TNL2K","model":"SPMTrack-G","rank_in_archive_order":2,"of":16,"metrics":{"AUC":"64.7","Normalized Precision":"82.6","precision":"70.6"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-tnl2k","task":"Visual Object Tracking","dataset":"TNL2K","model":"SPMTrack-L","rank_in_archive_order":3,"of":16,"metrics":{"AUC":"63.7","Normalized Precision":"81.5","precision":"69.2"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-tnl2k","task":"Visual Object Tracking","dataset":"TNL2K","model":"SPMTrack-B","rank_in_archive_order":7,"of":16,"metrics":{"AUC":"62.0","Normalized Precision":"79.7","precision":"66.7"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-trackingnet","task":"Visual Object Tracking","dataset":"TrackingNet","model":"SPMTrack-G","rank_in_archive_order":2,"of":40,"metrics":{"Accuracy":"87.3","Normalized Precision":"91.4","Precision":"88.1"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-trackingnet","task":"Visual Object Tracking","dataset":"TrackingNet","model":"SPMTrack-L","rank_in_archive_order":3,"of":40,"metrics":{"Accuracy":"86.9","Normalized Precision":"91","Precision":"87.2"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-trackingnet","task":"Visual Object Tracking","dataset":"TrackingNet","model":"SPMTrack-B","rank_in_archive_order":7,"of":40,"metrics":{"Accuracy":"86.1","Normalized Precision":"90.2","Precision":"85.6"},"uses_additional_data":false},{"leaderboard":"/sota/visual-object-tracking-on-uav123","task":"Visual Object Tracking","dataset":"UAV123","model":"SPMTrack-B","rank_in_archive_order":5,"of":16,"metrics":{"AUC":"0.717"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.18338","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.18338"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/WenRuiCai/SPMTrack","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"29d3c222b6dd6659","entry":"drop_add_residual_stochastic_depth","repo":"WenRuiCai/SPMTrack","repo_kind":"official","path":"trackit/models/methods/SPMTrack/modules/mixformer_attn.py","file_url":"https://github.com/WenRuiCai/SPMTrack/blob/HEAD/trackit/models/methods/SPMTrack/modules/mixformer_attn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"29d3c222b6dd6659"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}