{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/phystwin-physics-informed-reconstruction-and","title":"PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos","arxiv_id":"2503.17973","date":"2025-03-23","proceeding":null,"authors":["Hanxiao Jiang","Hao-Yu Hsu","Kaifeng Zhang","Hsin-Ni Yu","Shenlong Wang","Yunzhu Li"],"abstract":"Creating a physical digital twin of a real-world object has immense potential in robotics, content creation, and XR. In this paper, we present PhysTwin, a novel framework that uses sparse videos of dynamic objects under interaction to produce a photo- and physically realistic, real-time interactive virtual replica. Our approach centers on two key components: (1) a physics-informed representation that combines spring-mass models for realistic physical simulation, generative shape models for geometry, and Gaussian splats for rendering; and (2) a novel multi-stage, optimization-based inverse modeling framework that reconstructs complete geometry, infers dense physical properties, and replicates realistic appearance from videos. Our method integrates an inverse physics framework with visual perception cues, enabling high-fidelity reconstruction even from partial, occluded, and limited viewpoints. PhysTwin supports modeling various deformable objects, including ropes, stuffed animals, cloth, and delivery packages. Experiments show that PhysTwin outperforms competing methods in reconstruction, rendering, future prediction, and simulation under novel interactions. We further demonstrate its applications in interactive real-time simulation and model-based robotic motion planning.","url_abs":"https://arxiv.org/abs/2503.17973v1","url_pdf":"https://arxiv.org/pdf/2503.17973v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"phystwin-physics-informed-reconstruction-and","repo_url":"https://github.com/Jianghanxiao/PhysTwin","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"4d-reconstruction","task_name":"4D reconstruction"},{"task_slug":"deformable-object-manipulation","task_name":"Deformable Object Manipulation"},{"task_slug":"future-prediction","task_name":"Future prediction"},{"task_slug":"motion-planning","task_name":"Motion Planning"},{"task_slug":"robot-manipulation","task_name":"Robot Manipulation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.17973","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.17973"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Jianghanxiao/PhysTwin","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"listed":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e6771cd0bc38aa6b","entry":"evaluate_prediction","repo":"Jianghanxiao/PhysTwin","repo_kind":"listed","path":"evaluate_track.py","file_url":"https://github.com/Jianghanxiao/PhysTwin/blob/HEAD/evaluate_track.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e6771cd0bc38aa6b"}},{"code_sha256_prefix":"eaf2c846c5334d99","entry":"get_ray_directions","repo":"Jianghanxiao/PhysTwin","repo_kind":"listed","path":"gs_render.py","file_url":"https://github.com/Jianghanxiao/PhysTwin/blob/HEAD/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eaf2c846c5334d99"}},{"code_sha256_prefix":"74c6a993a0948239","entry":"prepare_output_and_logger","repo":"Jianghanxiao/PhysTwin","repo_kind":"listed","path":"gs_train.py","file_url":"https://github.com/Jianghanxiao/PhysTwin/blob/HEAD/gs_train.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74c6a993a0948239"}},{"code_sha256_prefix":"0e5b169606730db8","entry":"remove_gaussians_with_low_opacity","repo":"Jianghanxiao/PhysTwin","repo_kind":"listed","path":"gs_render.py","file_url":"https://github.com/Jianghanxiao/PhysTwin/blob/HEAD/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0e5b169606730db8"}},{"code_sha256_prefix":"50668bec9e524d63","entry":"remove_gaussians_with_mask","repo":"Jianghanxiao/PhysTwin","repo_kind":"listed","path":"gs_render.py","file_url":"https://github.com/Jianghanxiao/PhysTwin/blob/HEAD/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"50668bec9e524d63"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}