{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-learning-of-lagrangian-dynamics","title":"Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Control","arxiv_id":"2007.01926","date":"2020-07-03","proceeding":"NeurIPS 2020 12","authors":["Yaofeng Desmond Zhong","Naomi Ehrich Leonard"],"abstract":"Recent approaches for modelling dynamics of physical systems with neural networks enforce Lagrangian or Hamiltonian structure to improve prediction and generalization. However, when coordinates are embedded in high-dimensional data such as images, these approaches either lose interpretability or can only be applied to one particular example. We introduce a new unsupervised neural network model that learns Lagrangian dynamics from images, with interpretability that benefits prediction and control. The model infers Lagrangian dynamics on generalized coordinates that are simultaneously learned with a coordinate-aware variational autoencoder (VAE). The VAE is designed to account for the geometry of physical systems composed of multiple rigid bodies in the plane. By inferring interpretable Lagrangian dynamics, the model learns physical system properties, such as kinetic and potential energy, which enables long-term prediction of dynamics in the image space and synthesis of energy-based controllers.","url_abs":"https://arxiv.org/abs/2007.01926v3","url_pdf":"https://arxiv.org/pdf/2007.01926v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-learning-of-lagrangian-dynamics","repo_url":"https://github.com/DesmondZhong/Lagrangian_caVAE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.01926","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.01926"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/DesmondZhong/Lagrangian_caVAE","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"845f1457c5160418","entry":"Lag_Net","repo":"desmondzhong/lagrangian_cavae","repo_kind":"official","path":"lag_caVAE/lag.py","file_url":"https://github.com/desmondzhong/lagrangian_cavae/blob/HEAD/lag_caVAE/lag.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"845f1457c5160418"}},{"code_sha256_prefix":"c29fe5f041c08948","entry":"arrange_data","repo":"DesmondZhong/Lagrangian_caVAE","repo_kind":"official","path":"utils.py","file_url":"https://github.com/DesmondZhong/Lagrangian_caVAE/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c29fe5f041c08948"}},{"code_sha256_prefix":"0262341cbe5b7b15","entry":"choose_nonlinearity","repo":"DesmondZhong/Lagrangian_caVAE","repo_kind":"official","path":"lag_caVAE/nn_models.py","file_url":"https://github.com/DesmondZhong/Lagrangian_caVAE/blob/HEAD/lag_caVAE/nn_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0262341cbe5b7b15"}},{"code_sha256_prefix":"a72df854d424a529","entry":"from_pickle","repo":"DesmondZhong/Lagrangian_caVAE","repo_kind":"official","path":"utils.py","file_url":"https://github.com/DesmondZhong/Lagrangian_caVAE/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a72df854d424a529"}},{"code_sha256_prefix":"b90ef6af0c60c879","entry":"my_collate","repo":"DesmondZhong/Lagrangian_caVAE","repo_kind":"official","path":"utils.py","file_url":"https://github.com/DesmondZhong/Lagrangian_caVAE/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b90ef6af0c60c879"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}