{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2508-08612","title":"Hierarchical Visual Prompt Learning for Continual Video Instance Segmentation","arxiv_id":"2508.08612","date":"2025-08-12","proceeding":"ICCV","authors":["Jiahua Dong","Hui Yin","Wenqi Liang","Hanbin Zhao","Henghui Ding","Nicu Sebe","Salman Khan","Fahad Shahbaz Khan"],"abstract":"Video instance segmentation (VIS) has gained significant attention for its capability in tracking and segmenting object instances across video frames. However, most of the existing VIS approaches unrealistically assume that the categories of object instances remain fixed over time. Moreover, they experience catastrophic forgetting of old classes when required to continuously learn object instances belonging to new categories. To resolve these challenges, we develop a novel Hierarchical Visual Prompt Learning (HVPL) model that overcomes catastrophic forgetting of previous categories from both frame-level and video-level perspectives. Specifically, to mitigate forgetting at the frame level, we devise a task-specific frame prompt and an orthogonal gradient correction (OGC) module. The OGC module helps the frame prompt encode task-specific global instance information for new classes in each individual frame by projecting its gradients onto the orthogonal feature space of old classes. Furthermore, to address forgetting at the video level, we design a task-specific video prompt and a video context decoder. This decoder first embeds structural inter-class relationships across frames into the frame prompt features, and then propagates task-specific global video contexts from the frame prompt features to the video prompt. Through rigorous comparisons, our HVPL model proves to be more effective than baseline approaches. The code is available at https://github.com/JiahuaDong/HVPL.","url_abs":"https://arxiv.org/abs/2508.08612","url_pdf":"https://arxiv.org/pdf/2508.08612","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2508.08612","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2508.08612"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/JiahuaDong/HVPL","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"unverified":9},"by_repo_kind":{"found_in_text":{"samples":11,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f5dee2f80ed7988a","entry":"build_act_layer","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/transformer_decoder/grootv.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/transformer_decoder/grootv.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f5dee2f80ed7988a"}},{"code_sha256_prefix":"e7f1a63c2e99c70f","entry":"build_norm_layer","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/transformer_decoder/grootv.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/transformer_decoder/grootv.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7f1a63c2e99c70f"}},{"code_sha256_prefix":"bc2cb481a75c370d","entry":"batch_dice_loss","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/vita_matcher.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/vita_matcher.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bc2cb481a75c370d"}},{"code_sha256_prefix":"3061e23c21a52fd9","entry":"batch_index_opr","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/transformer_decoder/tree_scanning.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/transformer_decoder/tree_scanning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3061e23c21a52fd9"}},{"code_sha256_prefix":"1edd24985036b0bf","entry":"batch_sigmoid_ce_loss","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/vita_matcher.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/vita_matcher.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1edd24985036b0bf"}},{"code_sha256_prefix":"89f75e54ff128be0","entry":"dice_loss","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/vita_criterion.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/vita_criterion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"89f75e54ff128be0"}},{"code_sha256_prefix":"47bff92e02f2844a","entry":"interpolate","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/utils/misc.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47bff92e02f2844a"}},{"code_sha256_prefix":"58cc9ff3bf75e753","entry":"nested_tensor_from_tensor_list","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/utils/misc.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"58cc9ff3bf75e753"}},{"code_sha256_prefix":"d0c61e8dba511aa3","entry":"sigmoid_ce_loss","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/vita_criterion.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/vita_criterion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0c61e8dba511aa3"}},{"code_sha256_prefix":"f56d065ac11dd808","entry":"sigmoid_focal_loss","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/vita_criterion.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/vita_criterion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f56d065ac11dd808"}},{"code_sha256_prefix":"6a73429ffc79aa00","entry":"sigmoid_to_logit","repo":"JiahuaDong/HVPL","repo_kind":"found_in_text","path":"hvpl/modeling/transformer_decoder/vita.py","file_url":"https://github.com/JiahuaDong/HVPL/blob/HEAD/hvpl/modeling/transformer_decoder/vita.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6a73429ffc79aa00"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}