Papers › LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentation
LoSh: Long-Short Text Joint Prediction Network for Referring Video Object Segmentation
Linfeng Yuan, Miaojing Shi, Zijie Yue, Qijun Chen
Referring video object segmentation (RVOS) aims to segment the target instance referred by a given text expression in a video clip. The text expression normally contains sophisticated description of the instance's appearance, action, and relation with others. It is therefore rather difficult for a RVOS model to capture all these attributes correspondingly in the video; in fact, the model often favours more on the action- and relation-related visual attributes of the instance. This can end up with partial or even incorrect mask prediction of the target instance. We tackle this problem by taking a subject-centric short text expression from the original long text expression. The short one retains only the appearance-related information of the target instance so that we can use it to focus the model's attention on the instance's appearance. We let the model make joint predictions using both long and short text expressions; and insert a long-short cross-attention module to interact the joint features and a long-short predictions intersection loss to regulate the joint predictions. Besides the improvement on the linguistic part, we also introduce a forward-backward visual consistency loss, which utilizes optical flows to warp visual features between the annotated frames and their temporal neighbors for consistency. We build our method on top of two state of the art pipelines. Extensive experiments on A2D-Sentences, Refer-YouTube-VOS, JHMDB-Sentences and Refer-DAVIS17 show impressive improvements of our method.Code is available at https://github.com/LinfengYuan1997/Losh.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Referring Expression Segmentation | Refer-YouTube-VOS (2021 public validation) | LoSh-R | F | 66.0 | #19 of 33 | Archive leaderboard | report |
| Referring Expression Segmentation | Refer-YouTube-VOS (2021 public validation) | LoSh-R | J | 62.5 | #19 of 33 | Archive leaderboard | report |
| Referring Expression Segmentation | Refer-YouTube-VOS (2021 public validation) | LoSh-R | J&F | 64.2 | #19 of 33 | Archive leaderboard | report |
| Referring Video Object Segmentation | Ref-DAVIS17 | LoSh | F | 66.8 | #6 of 11 | Archive leaderboard | report |
| Referring Video Object Segmentation | Ref-DAVIS17 | LoSh | J | 61.8 | #6 of 11 | Archive leaderboard | report |
| Referring Video Object Segmentation | Ref-DAVIS17 | LoSh | J&F | 64.3 | #6 of 11 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections