{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-deep-moving-camera-background-model","title":"A Deep Moving-camera Background Model","arxiv_id":"2209.07923","date":"2022-09-16","proceeding":null,"authors":["Guy Erez","Ron Shapira Weber","Oren Freifeld"],"abstract":"In video analysis, background models have many applications such as background/foreground separation, change detection, anomaly detection, tracking, and more. However, while learning such a model in a video captured by a static camera is a fairly-solved task, in the case of a Moving-camera Background Model (MCBM), the success has been far more modest due to algorithmic and scalability challenges that arise due to the camera motion. Thus, existing MCBMs are limited in their scope and their supported camera-motion types. These hurdles also impeded the employment, in this unsupervised task, of end-to-end solutions based on deep learning (DL). Moreover, existing MCBMs usually model the background either on the domain of a typically-large panoramic image or in an online fashion. Unfortunately, the former creates several problems, including poor scalability, while the latter prevents the recognition and leveraging of cases where the camera revisits previously-seen parts of the scene. This paper proposes a new method, called DeepMCBM, that eliminates all the aforementioned issues and achieves state-of-the-art results. Concretely, first we identify the difficulties associated with joint alignment of video frames in general and in a DL setting in particular. Next, we propose a new strategy for joint alignment that lets us use a spatial transformer net with neither a regularization nor any form of specialized (and non-differentiable) initialization. Coupled with an autoencoder conditioned on unwarped robust central moments (obtained from the joint alignment), this yields an end-to-end regularization-free MCBM that supports a broad range of camera motions and scales gracefully. We demonstrate DeepMCBM's utility on a variety of videos, including ones beyond the scope of other methods. Our code is available at https://github.com/BGU-CS-VIL/DeepMCBM .","url_abs":"https://arxiv.org/abs/2209.07923v1","url_pdf":"https://arxiv.org/pdf/2209.07923v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-deep-moving-camera-background-model","repo_url":"https://github.com/bgu-cs-vil/deepmcbm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"change-detection","task_name":"Change Detection"},{"task_slug":"video-background-subtraction","task_name":"Video Background Subtraction"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"spatial-transformer","method_name":"Spatial Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/video-background-subtraction-on-davis-2017","task":"Video Background Subtraction","dataset":"DAVIS 2017 (bmx-trees)","model":"DeepMCBM (Basic/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.916"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-1","task":"Video Background Subtraction","dataset":"DAVIS 2017 (boxing-fisheye)","model":"DeepMCBM (Basic/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.927"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-2","task":"Video Background Subtraction","dataset":"DAVIS 2017 (breakdance-flare)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.963"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-3","task":"Video Background Subtraction","dataset":"DAVIS 2017 (continuousPan)","model":"DeepMCBM (CAE/Aff)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.94"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-4","task":"Video Background Subtraction","dataset":"DAVIS 2017 (dog-gooses)","model":"DeepMCBM (CAE/Aff)","rank_in_archive_order":1,"of":2,"metrics":{"AUC":"0.984"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-4","task":"Video Background Subtraction","dataset":"DAVIS 2017 (dog-gooses)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":2,"of":2,"metrics":{"AUC":"0.984"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-5","task":"Video Background Subtraction","dataset":"DAVIS 2017 (flamingo)","model":"DeepMCBM (CAE/Aff)","rank_in_archive_order":1,"of":2,"metrics":{"AUC":"0.98"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-5","task":"Video Background Subtraction","dataset":"DAVIS 2017 (flamingo)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":2,"of":2,"metrics":{"AUC":"0.98"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-6","task":"Video Background Subtraction","dataset":"DAVIS 2017 (horsejump-high)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.943"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-7","task":"Video Background Subtraction","dataset":"DAVIS 2017 (sidewalk)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.932"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-8","task":"Video Background Subtraction","dataset":"DAVIS 2017 (stroller)","model":"DeepMCBM (Basic/Aff)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.877"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-9","task":"Video Background Subtraction","dataset":"DAVIS 2017 (stunt)","model":"DeepMCBM (CAE/Aff)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.979"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-10","task":"Video Background Subtraction","dataset":"DAVIS 2017 (swing)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.897"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-11","task":"Video Background Subtraction","dataset":"DAVIS 2017 (tennis)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":1,"of":1,"metrics":{"AUC":"0.963"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-12","task":"Video Background Subtraction","dataset":"DAVIS 2017 (zoomInZoomOut)","model":"DeepMCBM (CAE/Aff)","rank_in_archive_order":1,"of":2,"metrics":{"AUC":"0.994"},"uses_additional_data":false},{"leaderboard":"/sota/video-background-subtraction-on-davis-2017-12","task":"Video Background Subtraction","dataset":"DAVIS 2017 (zoomInZoomOut)","model":"DeepMCBM (CAE/Hom)","rank_in_archive_order":2,"of":2,"metrics":{"AUC":"0.994"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.07923","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.07923"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bgu-cs-vil/deepmcbm","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/BGU-CS-VIL/DeepMCBM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":5,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":10,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"75786188ad2fbe6e","entry":"affine_warp","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75786188ad2fbe6e"}},{"code_sha256_prefix":"743452a3eb2ee8c1","entry":"create_pixel_stack","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"743452a3eb2ee8c1"}},{"code_sha256_prefix":"f2974723c2e7d212","entry":"get_grids","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2974723c2e7d212"}},{"code_sha256_prefix":"ad403c0a82e10079","entry":"homography_warp","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad403c0a82e10079"}},{"code_sha256_prefix":"55a9adb3d42575f9","entry":"trim_pixel_stack","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"55a9adb3d42575f9"}},{"code_sha256_prefix":"57a0fe3493fb5ce4","entry":"vec_to_perpective_matrix","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57a0fe3493fb5ce4"}},{"code_sha256_prefix":"65eb6ff441527c87","entry":"warp_inv","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65eb6ff441527c87"}},{"code_sha256_prefix":"2cc6784d503beafa","entry":"weighted_average","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2cc6784d503beafa"}},{"code_sha256_prefix":"c4a327e70cd8548f","entry":"BMN","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c4a327e70cd8548f"}},{"code_sha256_prefix":"fac5bd4846e7a1b5","entry":"trim_pixel_stack_","repo":"BGU-CS-VIL/DeepMCBM","repo_kind":"official","path":"src/BMN.py","file_url":"https://github.com/BGU-CS-VIL/DeepMCBM/blob/HEAD/src/BMN.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fac5bd4846e7a1b5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}