{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mbw-multi-view-bootstrapping-in-the-wild","title":"MBW: Multi-view Bootstrapping in the Wild","arxiv_id":"2210.01721","date":"2022-10-04","proceeding":null,"authors":["Mosam Dabhi","Chaoyang Wang","Tim Clifford","Laszlo Attila Jeni","Ian R. Fasel","Simon Lucey"],"abstract":"Labeling articulated objects in unconstrained settings have a wide variety of applications including entertainment, neuroscience, psychology, ethology, and many fields of medicine. Large offline labeled datasets do not exist for all but the most common articulated object categories (e.g., humans). Hand labeling these landmarks within a video sequence is a laborious task. Learned landmark detectors can help, but can be error-prone when trained from only a few examples. Multi-camera systems that train fine-grained detectors have shown significant promise in detecting such errors, allowing for self-supervised solutions that only need a small percentage of the video sequence to be hand-labeled. The approach, however, is based on calibrated cameras and rigid geometry, making it expensive, difficult to manage, and impractical in real-world scenarios. In this paper, we address these bottlenecks by combining a non-rigid 3D neural prior with deep flow to obtain high-fidelity landmark estimates from videos with only two or three uncalibrated, handheld cameras. With just a few annotations (representing 1-2% of the frames), we are able to produce 2D results comparable to state-of-the-art fully supervised methods, along with 3D reconstructions that are impossible with other existing approaches. Our Multi-view Bootstrapping in the Wild (MBW) approach demonstrates impressive results on standard human datasets, as well as tigers, cheetahs, fish, colobus monkeys, chimpanzees, and flamingos from videos captured casually in a zoo. We release the codebase for MBW as well as this challenging zoo dataset consisting image frames of tail-end distribution categories with their corresponding 2D, 3D labels generated from minimal human intervention.","url_abs":"https://arxiv.org/abs/2210.01721v1","url_pdf":"https://arxiv.org/pdf/2210.01721v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mbw-multi-view-bootstrapping-in-the-wild","repo_url":"https://github.com/mosamdabhi/mbw","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"mbw-multi-view-bootstrapping-in-the-wild","repo_url":"https://github.com/mosamdabhi/mbw-data","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"semi-supervised-2d-and-3d-landmark-labeling","task_name":"Semi-supervised 2D and 3D landmark labeling"},{"task_slug":"unsupervised-landmark-detection","task_name":"Unsupervised Landmark Detection"}],"methods":[],"datasets_introduced":[{"slug":"mbw-zoo-dataset","name":"MBW - Zoo Dataset","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2210.01721","atlas_url":"https://app.syntology.ai/?focus=2210.01721","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.01721"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/mosamdabhi/MBW-Data","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mosamdabhi/mbw-data","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mosamdabhi/mbw","reach":null}],"summary":{"ran":5,"ran_fixture":4,"ran_draft_wrong":5,"unverified":2},"by_repo_kind":{"official":{"samples":16,"ran":14,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c4b9ae62db86677e","entry":"BlockSparseCodingLayer","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"c4b9ae62db86677e"}},{"code_sha256_prefix":"5f29a57e42248b4a","entry":"OnPEstimator","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5f29a57e42248b4a"}},{"code_sha256_prefix":"95698fa3240b78e8","entry":"SparseCodeExtractionLayer","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"95698fa3240b78e8"}},{"code_sha256_prefix":"7c015219ac5af148","entry":"VecSparseCodingLayer","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"7c015219ac5af148"}},{"code_sha256_prefix":"d8ceaef95e13792e","entry":"computeMPJPE","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"d8ceaef95e13792e"}},{"code_sha256_prefix":"35f88ab6c63924c0","entry":"error_ratio","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"35f88ab6c63924c0"}},{"code_sha256_prefix":"dd8dc8b64c5e304a","entry":"get_device","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"dd8dc8b64c5e304a"}},{"code_sha256_prefix":"87df747715c00040","entry":"get_joints","repo":"mosamdabhi/mbw-data","repo_kind":"official","path":"python/joints_function.py","file_url":"https://github.com/mosamdabhi/mbw-data/blob/HEAD/python/joints_function.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"87df747715c00040"}},{"code_sha256_prefix":"5b931698793ecf7b","entry":"make_rotation_matrix","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5b931698793ecf7b"}},{"code_sha256_prefix":"9edb93f56e50a211","entry":"normalize_3d_structure","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"9edb93f56e50a211"}},{"code_sha256_prefix":"849a1f3486ab123f","entry":"onp","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"849a1f3486ab123f"}},{"code_sha256_prefix":"5fa5784ee4bc9ff4","entry":"process_data_type","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"5fa5784ee4bc9ff4"}},{"code_sha256_prefix":"cb3424ad9ae6d988","entry":"procrustes_align","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cb3424ad9ae6d988"}},{"code_sha256_prefix":"fbb59c9ab4d488e3","entry":"relu_threshold","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fbb59c9ab4d488e3"}},{"code_sha256_prefix":"881edbe731b97896","entry":"MVNRSfM","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"881edbe731b97896"}},{"code_sha256_prefix":"00840fbf82519fc0","entry":"make_dir","repo":"mosamdabhi/mbw","repo_kind":"official","path":"modules/mvnrsfm/mvnrsfm_train.py","file_url":"https://github.com/mosamdabhi/mbw/blob/HEAD/modules/mvnrsfm/mvnrsfm_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"00840fbf82519fc0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}