{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-masks-from-boxes-by-mining-spatio","title":"Generating Masks from Boxes by Mining Spatio-Temporal Consistencies in Videos","arxiv_id":"2101.02196","date":"2021-01-06","proceeding":"ICCV 2021 10","authors":["Bin Zhao","Goutam Bhat","Martin Danelljan","Luc van Gool","Radu Timofte"],"abstract":"Segmenting objects in videos is a fundamental computer vision task. The current deep learning based paradigm offers a powerful, but data-hungry solution. However, current datasets are limited by the cost and human effort of annotating object masks in videos. This effectively limits the performance and generalization capabilities of existing video segmentation methods. To address this issue, we explore weaker form of bounding box annotations. We introduce a method for generating segmentation masks from per-frame bounding box annotations in videos. To this end, we propose a spatio-temporal aggregation module that effectively mines consistencies in the object and background appearance across multiple frames. We use our resulting accurate masks for weakly supervised training of video object segmentation (VOS) networks. We generate segmentation masks for large scale tracking datasets, using only their bounding box annotations. The additional data provides substantially better generalization performance leading to state-of-the-art results in both the VOS and more challenging tracking domain.","url_abs":"https://arxiv.org/abs/2101.02196v1","url_pdf":"https://arxiv.org/pdf/2101.02196v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-masks-from-boxes-by-mining-spatio","repo_url":"https://github.com/visionml/pytracking","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"object","task_name":"Object"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"video-object-segmentation","task_name":"Video Object Segmentation"},{"task_slug":"video-segmentation","task_name":"Video Segmentation"},{"task_slug":"video-semantic-segmentation","task_name":"Video Semantic Segmentation"}],"methods":[{"method_slug":"vos","method_name":"VOS"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2101.02196","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2101.02196"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/visionml/pytracking","reach":null}],"summary":{"ran":2,"ran_fixture":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"22f4d8f301f72b01","entry":"BaseTracker","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"22f4d8f301f72b01"}},{"code_sha256_prefix":"e6fff242ca8417c0","entry":"TensorList","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e6fff242ca8417c0"}},{"code_sha256_prefix":"d850215766b052a8","entry":"numpy_to_torch","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d850215766b052a8"}},{"code_sha256_prefix":"92cc800ac9e750ca","entry":"sample_patch","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"92cc800ac9e750ca"}},{"code_sha256_prefix":"732ad9df2dc9acc2","entry":"sample_patch_multiscale","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"732ad9df2dc9acc2"}},{"code_sha256_prefix":"92f09d9cbd01c287","entry":"sample_patch_transformed","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"92f09d9cbd01c287"}},{"code_sha256_prefix":"b678f41a9a57b02d","entry":"LWL","repo":"visionml/pytracking","repo_kind":"official","path":"pytracking/tracker/lwl/lwl.py","file_url":"https://github.com/visionml/pytracking/blob/HEAD/pytracking/tracker/lwl/lwl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"b678f41a9a57b02d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}