{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/visevent-reliable-object-tracking-via","title":"VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows","arxiv_id":"2108.05015","date":"2021-08-11","proceeding":null,"authors":["Xiao Wang","Jianing Li","Lin Zhu","Zhipeng Zhang","Zhe Chen","Xin Li","YaoWei Wang","Yonghong Tian","Feng Wu"],"abstract":"Different from visible cameras which record intensity images frame by frame, the biologically inspired event camera produces a stream of asynchronous and sparse events with much lower latency. In practice, visible cameras can better perceive texture details and slow motion, while event cameras can be free from motion blurs and have a larger dynamic range which enables them to work well under fast motion and low illumination. Therefore, the two sensors can cooperate with each other to achieve more reliable object tracking. In this work, we propose a large-scale Visible-Event benchmark (termed VisEvent) due to the lack of a realistic and scaled dataset for this task. Our dataset consists of 820 video pairs captured under low illumination, high speed, and background clutter scenarios, and it is divided into a training and a testing subset, each of which contains 500 and 320 videos, respectively. Based on VisEvent, we transform the event flows into event images and construct more than 30 baseline methods by extending current single-modality trackers into dual-modality versions. More importantly, we further build a simple but effective tracking algorithm by proposing a cross-modality transformer, to achieve more effective feature fusion between visible and event data. Extensive experiments on the proposed VisEvent dataset, FE108, COESOT, and two simulated datasets (i.e., OTB-DVS and VOT-DVS), validated the effectiveness of our model. The dataset and source code have been released on: \\url{https://github.com/wangxiao5791509/VisEvent_SOT_Benchmark}.","url_abs":"https://arxiv.org/abs/2108.05015v4","url_pdf":"https://arxiv.org/pdf/2108.05015v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"visevent-reliable-object-tracking-via","repo_url":"https://github.com/wangxiao5791509/RGB-DVS-SOT-Baselines","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"visevent-reliable-object-tracking-via","repo_url":"https://github.com/wangxiao5791509/VisEvent_SOT_Benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"object-tracking","task_name":"Object Tracking"}],"methods":[],"datasets_introduced":[{"slug":"visevent","name":"VisEvent","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/object-tracking-on-visevent","task":"Object Tracking","dataset":"VisEvent","model":"RT-MDNet","rank_in_archive_order":1,"of":1,"metrics":{"Precision Plot":"0.627"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2108.05015","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2108.05015"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wangxiao5791509/VisEvent_SOT_Benchmark","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wangxiao5791509/RGB-DVS-SOT-Baselines","reach":{"status":"ok"}}],"summary":{"ran_honours":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fc53231bd0e0dd24","entry":"make_color_histo","repo":"wangxiao5791509/VisEvent_SOT_Benchmark","repo_kind":"official","path":"event2frame_code/1_davis_to_images.py","file_url":"https://github.com/wangxiao5791509/VisEvent_SOT_Benchmark/blob/HEAD/event2frame_code/1_davis_to_images.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc53231bd0e0dd24"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}