{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-grained-temporal-prototype-learning-for","title":"Multi-grained Temporal Prototype Learning for Few-shot Video Object Segmentation","arxiv_id":"2309.11160","date":"2023-09-20","proceeding":"ICCV 2023 1","authors":["Nian Liu","Kepan Nan","Wangbo Zhao","Yuanwei Liu","Xiwen Yao","Salman Khan","Hisham Cholakkal","Rao Muhammad Anwer","Junwei Han","Fahad Shahbaz Khan"],"abstract":"Few-Shot Video Object Segmentation (FSVOS) aims to segment objects in a query video with the same category defined by a few annotated support images. However, this task was seldom explored. In this work, based on IPMT, a state-of-the-art few-shot image segmentation method that combines external support guidance information with adaptive query guidance cues, we propose to leverage multi-grained temporal guidance information for handling the temporal correlation nature of video data. We decompose the query video information into a clip prototype and a memory prototype for capturing local and long-term internal temporal guidance, respectively. Frame prototypes are further used for each frame independently to handle fine-grained adaptive guidance and enable bidirectional clip-frame prototype communication. To reduce the influence of noisy memory, we propose to leverage the structural similarity relation among different predicted regions and the support for selecting reliable memory frames. Furthermore, a new segmentation loss is also proposed to enhance the category discriminability of the learned prototypes. Experimental results demonstrate that our proposed video IPMT model significantly outperforms previous models on two benchmark datasets. Code is available at https://github.com/nankepan/VIPMT.","url_abs":"https://arxiv.org/abs/2309.11160v1","url_pdf":"https://arxiv.org/pdf/2309.11160v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-grained-temporal-prototype-learning-for","repo_url":"https://github.com/nankepan/VIPMT","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.11160","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.11160"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/nankepan/VIPMT","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"df77cb77dfe58d6d","entry":"Weighted_GAP","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/VIPMT.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/VIPMT.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"df77cb77dfe58d6d"}},{"code_sha256_prefix":"54ee1aca8e2c2cf1","entry":"binary_entropy_loss","repo":"nankepan/VIPMT","repo_kind":"official","path":"libs/utils/loss.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/libs/utils/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"54ee1aca8e2c2cf1"}},{"code_sha256_prefix":"2a80220dabcb742a","entry":"conv1x1","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/resnet.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2a80220dabcb742a"}},{"code_sha256_prefix":"600ff2c45e0de056","entry":"conv3x3","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/resnet.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/resnet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"600ff2c45e0de056"}},{"code_sha256_prefix":"2d13347de310a810","entry":"cross_entropy_loss","repo":"nankepan/VIPMT","repo_kind":"official","path":"libs/utils/loss.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/libs/utils/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2d13347de310a810"}},{"code_sha256_prefix":"627804c56ccbcbea","entry":"mask_iou","repo":"nankepan/VIPMT","repo_kind":"official","path":"libs/utils/loss.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/libs/utils/loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"627804c56ccbcbea"}},{"code_sha256_prefix":"420141e2a84ccabc","entry":"meaniou","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/VIPMT.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/VIPMT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"420141e2a84ccabc"}},{"code_sha256_prefix":"168ed782bacd830e","entry":"overalliou","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/VIPMT.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/VIPMT.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"168ed782bacd830e"}},{"code_sha256_prefix":"dc824689b4f991cf","entry":"resnet18","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/resnet.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc824689b4f991cf"}},{"code_sha256_prefix":"4708c105ce1b6276","entry":"weighted_dice_loss","repo":"nankepan/VIPMT","repo_kind":"official","path":"model/loss.py","file_url":"https://github.com/nankepan/VIPMT/blob/HEAD/model/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4708c105ce1b6276"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}