{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/temporal-memory-attention-for-video-semantic","title":"Temporal Memory Attention for Video Semantic Segmentation","arxiv_id":"2102.08643","date":"2021-02-17","proceeding":null,"authors":["Hao Wang","Weining Wang","Jing Liu"],"abstract":"Video semantic segmentation requires to utilize the complex temporal relations between frames of the video sequence. Previous works usually exploit accurate optical flow to leverage the temporal relations, which suffer much from heavy computational cost. In this paper, we propose a Temporal Memory Attention Network (TMANet) to adaptively integrate the long-range temporal relations over the video sequence based on the self-attention mechanism without exhaustive optical flow prediction. Specially, we construct a memory using several past frames to store the temporal information of the current frame. We then propose a temporal memory attention module to capture the relation between the current frame and the memory to enhance the representation of the current frame. Our method achieves new state-of-the-art performances on two challenging video semantic segmentation datasets, particularly 80.3% mIoU on Cityscapes and 76.5% mIoU on CamVid with ResNet-50.","url_abs":"https://arxiv.org/abs/2102.08643v2","url_pdf":"https://arxiv.org/pdf/2102.08643v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"temporal-memory-attention-for-video-semantic","repo_url":"https://github.com/wanghao9610/TMANet","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-segmentation-on-urbanlf","task":"Semantic Segmentation","dataset":"UrbanLF","model":"TMANet","rank_in_archive_order":10,"of":14,"metrics":{"mIoU (Real)":"77.14","mIoU (Syn)":"76.41"},"uses_additional_data":false},{"leaderboard":"/sota/video-semantic-segmentation-on-camvid","task":"Video Semantic Segmentation","dataset":"CamVid","model":"TMANet-50","rank_in_archive_order":1,"of":6,"metrics":{"Mean IoU":"76.5"},"uses_additional_data":false},{"leaderboard":"/sota/video-semantic-segmentation-on-camvid","task":"Video Semantic Segmentation","dataset":"CamVid","model":"Netwarp","rank_in_archive_order":5,"of":6,"metrics":{"Mean IoU":"74.7"},"uses_additional_data":false},{"leaderboard":"/sota/video-semantic-segmentation-on-cityscapes-val","task":"Video Semantic Segmentation","dataset":"Cityscapes val","model":"TMANet-50","rank_in_archive_order":1,"of":9,"metrics":{"mIoU":"80.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2102.08643","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}