{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sc4d-sparse-controlled-video-to-4d-generation","title":"SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer","arxiv_id":"2404.03736","date":"2024-04-04","proceeding":null,"authors":["Zijie Wu","Chaohui Yu","Yanqin Jiang","Chenjie Cao","Fan Wang","Xiang Bai"],"abstract":"Recent advances in 2D/3D generative models enable the generation of dynamic 3D objects from a single-view video. Existing approaches utilize score distillation sampling to form the dynamic scene as dynamic NeRF or dense 3D Gaussians. However, these methods struggle to strike a balance among reference view alignment, spatio-temporal consistency, and motion fidelity under single-view conditions due to the implicit nature of NeRF or the intricate dense Gaussian motion prediction. To address these issues, this paper proposes an efficient, sparse-controlled video-to-4D framework named SC4D, that decouples motion and appearance to achieve superior video-to-4D generation. Moreover, we introduce Adaptive Gaussian (AG) initialization and Gaussian Alignment (GA) loss to mitigate shape degeneration issue, ensuring the fidelity of the learned motion and shape. Comprehensive experimental results demonstrate that our method surpasses existing methods in both quality and efficiency. In addition, facilitated by the disentangled modeling of motion and appearance of SC4D, we devise a novel application that seamlessly transfers the learned motion onto a diverse array of 4D entities according to textual descriptions.","url_abs":"https://arxiv.org/abs/2404.03736v2","url_pdf":"https://arxiv.org/pdf/2404.03736v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sc4d-sparse-controlled-video-to-4d-generation","repo_url":"https://github.com/jarrentwu1031/sc4d","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.03736","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.03736"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jarrentwu1031/sc4d","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":2,"ran":6,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5a8b1e0eb5ec2789","entry":"RGB2SH","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"sh_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/sh_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a8b1e0eb5ec2789"}},{"code_sha256_prefix":"d46f0f7778585fae","entry":"build_rotation","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"helpers.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d46f0f7778585fae"}},{"code_sha256_prefix":"d3d0dc69477aa06b","entry":"eval_sh","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"sh_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/sh_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3d0dc69477aa06b"}},{"code_sha256_prefix":"0582b921f88193d0","entry":"get_embedder","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"pos_enc.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/pos_enc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0582b921f88193d0"}},{"code_sha256_prefix":"6d2c15014d993b0e","entry":"get_expon_lr_func","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"gs_renderer.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/gs_renderer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6d2c15014d993b0e"}},{"code_sha256_prefix":"b488da571728b636","entry":"inverse_sigmoid","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"gs_renderer.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/gs_renderer.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b488da571728b636"}},{"code_sha256_prefix":"0ac483269bc441c5","entry":"params2rendervar","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"helpers.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/helpers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ac483269bc441c5"}},{"code_sha256_prefix":"8f65bf51b0f1b5e1","entry":"quaternion_to_matrix","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"deform_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/deform_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8f65bf51b0f1b5e1"}},{"code_sha256_prefix":"cd789cab1b072db4","entry":"strip_lowerdiag","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"gs_renderer.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/gs_renderer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cd789cab1b072db4"}},{"code_sha256_prefix":"55e5c8b4a3e6a864","entry":"SH2RGB","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"sh_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/sh_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55e5c8b4a3e6a864"}},{"code_sha256_prefix":"e35aafbf0069edb6","entry":"dot","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/cam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e35aafbf0069edb6"}},{"code_sha256_prefix":"43cdc168f88caf9b","entry":"length","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/cam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43cdc168f88caf9b"}},{"code_sha256_prefix":"af8d9befb172b0a1","entry":"produce_edge_matrix_nfmt","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"deform_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/deform_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af8d9befb172b0a1"}},{"code_sha256_prefix":"8a8484ccf552d87f","entry":"safe_normalize","repo":"jarrentwu1031/sc4d","repo_kind":"official","path":"cam_utils.py","file_url":"https://github.com/jarrentwu1031/sc4d/blob/HEAD/cam_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a8484ccf552d87f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}