{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/physics-as-inverse-graphics-joint","title":"Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video","arxiv_id":"1905.11169","date":"2019-05-27","proceeding":"ICLR 2020 1","authors":["Miguel Jaques","Michael Burke","Timothy Hospedales"],"abstract":"We propose a model that is able to perform unsupervised physical parameter estimation of systems from video, where the differential equations governing the scene dynamics are known, but labeled states or objects are not available. Existing physical scene understanding methods require either object state supervision, or do not integrate with differentiable physics to learn interpretable system parameters and states. We address this problem through a physics-as-inverse-graphics approach that brings together vision-as-inverse-graphics and differentiable physics engines, enabling objects and explicit state and velocity representations to be discovered. This framework allows us to perform long term extrapolative video prediction, as well as vision-based model-predictive control. Our approach significantly outperforms related unsupervised methods in long-term future frame prediction of systems with interacting objects (such as ball-spring or 3-body gravitational systems), due to its ability to build dynamics into the model as an inductive bias. We further show the value of this tight vision-physics integration by demonstrating data-efficient learning of vision-actuated model-based control for a pendulum system. We also show that the controller's interpretability provides unique capabilities in goal-driven control and physical reasoning for zero-data adaptation.","url_abs":"https://arxiv.org/abs/1905.11169v2","url_pdf":"https://arxiv.org/pdf/1905.11169v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"physics-as-inverse-graphics-joint","repo_url":"https://github.com/seuqaj114/paig","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"model-predictive-control","task_name":"Model Predictive Control"},{"task_slug":"scene-understanding","task_name":"Scene Understanding"},{"task_slug":"video-prediction","task_name":"Video Prediction"},{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1905.11169","atlas_url":"https://app.syntology.ai/?focus=1905.11169","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1905.11169"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/seuqaj114/paig","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":10},"by_repo_kind":{"listed":{"samples":13,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"59717137f2d75193","entry":"classes_in_module","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/utils/misc.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/utils/misc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59717137f2d75193"}},{"code_sha256_prefix":"21d9e9de327d2bc4","entry":"get_iterators","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/datasets/iterators.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/datasets/iterators.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21d9e9de327d2bc4"}},{"code_sha256_prefix":"9842b4e36ae755ef","entry":"verify_wall_collision","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/datasets/generators.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/datasets/generators.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9842b4e36ae755ef"}},{"code_sha256_prefix":"4c87ffad96ed1779","entry":"batch_transformer","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/network/stn.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/network/stn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c87ffad96ed1779"}},{"code_sha256_prefix":"77d0c6157590e526","entry":"compute_wall_collision","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/datasets/generators.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/datasets/generators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"77d0c6157590e526"}},{"code_sha256_prefix":"13bd75adaddef15f","entry":"gallery","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/utils/viz.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/utils/viz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13bd75adaddef15f"}},{"code_sha256_prefix":"51866c920be1c1de","entry":"generate_bouncing_ball_dataset","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/datasets/generators.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/datasets/generators.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51866c920be1c1de"}},{"code_sha256_prefix":"220c0b78e4f17c33","entry":"gif","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/utils/viz.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/utils/viz.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"220c0b78e4f17c33"}},{"code_sha256_prefix":"ca791728b69bd889","entry":"rgb2gray","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/utils/misc.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/utils/misc.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca791728b69bd889"}},{"code_sha256_prefix":"03d4a62c20d2bc26","entry":"shallow_unet","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/network/blocks.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/network/blocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03d4a62c20d2bc26"}},{"code_sha256_prefix":"c8e502f4fd41a29c","entry":"stn","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/network/stn.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/network/stn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8e502f4fd41a29c"}},{"code_sha256_prefix":"9499f44469865a2f","entry":"unet","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/network/blocks.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/network/blocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9499f44469865a2f"}},{"code_sha256_prefix":"4c5821e09609fdb0","entry":"variable_from_network","repo":"seuqaj114/paig","repo_kind":"listed","path":"nn/network/blocks.py","file_url":"https://github.com/seuqaj114/paig/blob/HEAD/nn/network/blocks.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c5821e09609fdb0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}