Papers › SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted...

SWEM: Towards Real-Time Video Object Segmentation with Sequential Weighted Expectation-Maximization

22 Aug 2022CVPR 2022 1arXiv:2208.10128archive 2025-07-28

Zhihui Lin, Tianyu Yang, Maomao Li, Ziyu Wang, Chun Yuan, Wenhao Jiang, Wei Liu

Matching-based methods, especially those based on space-time memory, are significantly ahead of other solutions in semi-supervised video object segmentation (VOS). However, continuously growing and redundant template features lead to an inefficient inference. To alleviate this, we propose a novel Sequential Weighted Expectation-Maximization (SWEM) network to greatly reduce the redundancy of memory features. Different from the previous methods which only detect feature redundancy between frames, SWEM merges both intra-frame and inter-frame similar features by leveraging the sequential weighted EM algorithm. Further, adaptive weights for frame features endow SWEM with the flexibility to represent hard samples, improving the discrimination of templates. Besides, the proposed method maintains a fixed number of template features in memory, which ensures the stable inference complexity of the VOS system. Extensive experiments on commonly used DAVIS and YouTube-VOS datasets verify the high efficiency (36 FPS) and high performance (84.3% 𝒥&ℱ on DAVIS 2017 validation dataset) of SWEM. Code is available at: https://github.com/lmm077/SWEM.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology: its repositories, every harvested function with whether it ran, its licence and the call to fetch it.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

lmm077/SWEM officialmentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Semantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic Segmentation

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D16 val (F) 89.0 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D16 val (G) 88.1 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D16 val (J) 87.3 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D17 val (F) 79.8 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D17 val (G) 77.2 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM D17 val (J) 74.5 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS (no YouTube-VOS training) SWEM FPS 36.0 #6 of 26 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2016 SWEM (val) F-measure (Mean) 89.0 #41 of 78 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2016 SWEM (val) J&F 88.1 #41 of 78 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2016 SWEM (val) Jaccard (Mean) 87.3 #41 of 78 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2016 SWEM (val) Speed (FPS) 36 #41 of 78 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) SWEM F-measure (Mean) 79.8 #51 of 81 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) SWEM J&F 77.2 #51 of 81 Archive leaderboard report
Semi-Supervised Video Object Segmentation DAVIS 2017 (val) SWEM Jaccard (Mean) 74.5 #51 of 81 Archive leaderboard report
Semi-Supervised Video Object Segmentation MOSE SWEM F 54.9 #15 of 17 Archive leaderboard report
Semi-Supervised Video Object Segmentation MOSE SWEM J 46.8 #15 of 17 Archive leaderboard report
Semi-Supervised Video Object Segmentation MOSE SWEM J&F 50.9 #15 of 17 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

VOS

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections