Papers › Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network
Video Saliency Prediction Using Enhanced Spatiotemporal Alignment Network
Jin Chen, Huihui Song, Kaihua Zhang, Bo Liu, Qingshan Liu
Due to a variety of motions across different frames, it is highly challenging to learn an effective spatiotemporal representation for accurate video saliency prediction (VSP). To address this issue, we develop an effective spatiotemporal feature alignment network tailored to VSP, mainly including two key sub-networks: a multi-scale deformable convolutional alignment network (MDAN) and a bidirectional convolutional Long Short-Term Memory (Bi-ConvLSTM) network. The MDAN learns to align the features of the neighboring frames to the reference one in a coarse-to-fine manner, which can well handle various motions. Specifically, the MDAN owns a pyramidal feature hierarchy structure that first leverages deformable convolution (Dconv) to align the lower-resolution features across frames, and then aggregates the aligned features to align the higher-resolution features, progressively enhancing the features from top to bottom. The output of MDAN is then fed into the Bi-ConvLSTM for further enhancement, which captures the useful long-time temporal information along forward and backward timing directions to effectively guide attention orientation shift prediction under complex scene transformation. Finally, the enhanced features are decoded to generate the predicted saliency map. The proposed model is trained end-to-end without any intricate post processing. Extensive evaluations on four VSP benchmark datasets demonstrate that the proposed method achieves favorable performance against state-of-the-art methods. The source codes and all the results will be released.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | AUC-J | 0.841 | #7 of 14 | Archive leaderboard | report |
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | CC | 0.665 | #7 of 14 | Archive leaderboard | report |
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | FPS | 3.35 | #7 of 14 | Archive leaderboard | report |
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | KLDiv | 0.583 | #7 of 14 | Archive leaderboard | report |
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | NSS | 1.81 | #7 of 14 | Archive leaderboard | report |
| Video Saliency Detection | MSU Video Saliency Prediction | STRA-Net | SIM | 0.591 | #7 of 14 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections