{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-physics-from-video-unsupervised","title":"Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems","arxiv_id":"2410.01376","date":"2024-10-02","proceeding":"CVPR 2025 1","authors":["Alejandro Castañeda Garcia","Jan van Gemert","Daan Brinks","Nergis Tömen"],"abstract":"Extracting physical dynamical system parameters from recorded observations is key in natural science. Current methods for automatic parameter estimation from video train supervised deep networks on large datasets. Such datasets require labels, which are difficult to acquire. While some unsupervised techniques--which depend on frame prediction--exist, they suffer from long training times, initialization instabilities, only consider motion-based dynamical systems, and are evaluated mainly on synthetic data. In this work, we propose an unsupervised method to estimate the physical parameters of known, continuous governing equations from single videos suitable for different dynamical systems beyond motion and robust to initialization. Moreover, we remove the need for frame prediction by implementing a KL-divergence-based loss function in the latent space, which avoids convergence to trivial solutions and reduces model size and compute. We first evaluate our model on synthetic data, as commonly done. After which, we take the field closer to reality by recording Delfys75: our own real-world dataset of 75 videos for five different types of dynamical systems to evaluate our method and others. Our method compares favorably to others. %, yet, and real-world video datasets and demonstrate improved parameter estimation accuracy compared to existing methods. Code and data are available online:https://github.com/Alejandro-neuro/Learning_physics_from_video.","url_abs":"https://arxiv.org/abs/2410.01376v2","url_pdf":"https://arxiv.org/pdf/2410.01376v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-physics-from-video-unsupervised","repo_url":"https://github.com/alejandro-neuro/learning_physics_from_video","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.01376","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01376"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alejandro-neuro/learning_physics_from_video","reach":{"status":"ok"}}],"summary":{"ran":7,"unverified":3},"by_repo_kind":{"official":{"samples":10,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"cc329f6ed02990a3","entry":"KL_divergence","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/loss_func.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/loss_func.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc329f6ed02990a3"}},{"code_sha256_prefix":"8e95b679141701b6","entry":"evaluate_epoch","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/train.txt.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/train.txt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8e95b679141701b6"}},{"code_sha256_prefix":"4d98a8807f5d6fba","entry":"freeze","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/train.txt.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/train.txt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d98a8807f5d6fba"}},{"code_sha256_prefix":"17cebb883a7d310c","entry":"getLoader_folder","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/loader.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/loader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17cebb883a7d310c"}},{"code_sha256_prefix":"1a9cb209c11cea5b","entry":"latent_loss","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/loss_func.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/loss_func.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1a9cb209c11cea5b"}},{"code_sha256_prefix":"c4226f0238390646","entry":"mse_loss","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/loss_func.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/loss_func.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c4226f0238390646"}},{"code_sha256_prefix":"51302b801775d0c4","entry":"train_epoch","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/train.txt.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/train.txt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51302b801775d0c4"}},{"code_sha256_prefix":"87917ca1e413167c","entry":"getModel","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/models/PhysModels.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/models/PhysModels.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87917ca1e413167c"}},{"code_sha256_prefix":"f0a2125a58793c80","entry":"getOptimizer","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/optimizer_Factory.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/optimizer_Factory.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f0a2125a58793c80"}},{"code_sha256_prefix":"d21fd18179e10d84","entry":"process_video","repo":"alejandro-neuro/learning_physics_from_video","repo_kind":"official","path":"src/utils/video2npy.py","file_url":"https://github.com/alejandro-neuro/learning_physics_from_video/blob/HEAD/src/utils/video2npy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d21fd18179e10d84"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}