{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-rigidity-in-dynamic-scenes-with-a","title":"Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation","arxiv_id":"1804.04259","date":"2018-04-12","proceeding":"ECCV 2018 9","authors":["Zhaoyang Lv","Kihwan Kim","Alejandro Troccoli","Deqing Sun","James M. Rehg","Jan Kautz"],"abstract":"Estimation of 3D motion in a dynamic scene from a temporal pair of images is\na core task in many scene understanding problems. In real world applications, a\ndynamic scene is commonly captured by a moving camera (i.e., panning, tilting\nor hand-held), increasing the task complexity because the scene is observed\nfrom different view points. The main challenge is the disambiguation of the\ncamera motion from scene motion, which becomes more difficult as the amount of\nrigidity observed decreases, even with successful estimation of 2D image\ncorrespondences. Compared to other state-of-the-art 3D scene flow estimation\nmethods, in this paper we propose to \\emph{learn} the rigidity of a scene in a\nsupervised manner from a large collection of dynamic scene data, and directly\ninfer a rigidity mask from two sequential images with depths. With the learned\nnetwork, we show how we can effectively estimate camera motion and projected\nscene flow using computed 2D optical flow and the inferred rigidity mask. For\ntraining and testing the rigidity network, we also provide a new semi-synthetic\ndynamic scene dataset (synthetic foreground objects with a real background) and\nan evaluation split that accounts for the percentage of observed non-rigid\npixels. Through our evaluation we show the proposed framework outperforms\ncurrent state-of-the-art scene flow estimation methods in challenging dynamic\nscenes.","url_abs":"http://arxiv.org/abs/1804.04259v2","url_pdf":"http://arxiv.org/pdf/1804.04259v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-rigidity-in-dynamic-scenes-with-a","repo_url":"https://github.com/NVlabs/learningrigidity","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"scene-flow-estimation","task_name":"Scene Flow Estimation"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1804.04259","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}