{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/polar-relative-positional-encoding-for-video","title":"Polar Relative Positional Encoding for Video-Language Segmentation","arxiv_id":null,"date":"2020-07-20","proceeding":null,"authors":["Ke Ning","Lingxi Xie","Fei Wu","Qi Tian"],"abstract":"In this paper, we tackle a challenging task named video-language segmentation. Given a video and a sentence in natural language, the goal is to segment the object or actor described by the sentence in video frames. To accurately denote a target object, the given sentence usually refers to multiple attributes, such as nearby objects with spatial relations, etc. In this paper, we propose a novel Polar Relative Positional Encoding (PRPE) mechanism that represents spatial relations in a ``linguistic'' way, i.e., in terms of direction and range. Sentence feature can interact with positional embeddings in a more direct way to extract the implied relative positional relations. We also propose parameterized functions for these positional embeddings to adapt real-value directions and ranges. With PRPE, we design a Polar Attention Module (PAM) as the basic module for vision-language fusion. Our method outperforms previous best method by a large margin of 11.4% absolute improvement in terms of mAP on the challenging A2D Sentences dataset. Our method also achieves competitive performances on the J-HMDB Sentences dataset.","url_abs":"https://www.ijcai.org/proceedings/2020/132","url_pdf":"https://www.ijcai.org/proceedings/2020/0132.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"referring-expression-segmentation","task_name":"Referring Expression Segmentation"},{"task_slug":"sentence","task_name":"Sentence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/referring-expression-segmentation-on-a2d","task":"Referring Expression Segmentation","dataset":"A2D Sentences","model":"PRPE","rank_in_archive_order":14,"of":27,"metrics":{"AP":"0.388","IoU mean":"0.529","IoU overall":"0.661","Precision@0.5":"0.634","Precision@0.6":"0.579","Precision@0.7":"0.483","Precision@0.8":"0.322","Precision@0.9":"0.083"},"uses_additional_data":false},{"leaderboard":"/sota/referring-expression-segmentation-on-j-hmdb","task":"Referring Expression Segmentation","dataset":"J-HMDB","model":"PRPE","rank_in_archive_order":11,"of":21,"metrics":{"AP":"0.294","Precision@0.5":"0.572","Precision@0.6":"0.690","Precision@0.7":"0.319","Precision@0.8":"0.06","Precision@0.9":"0.001"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}