{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uni3c-unifying-precisely-3d-enhanced-camera","title":"Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation","arxiv_id":"2504.14899","date":"2025-04-21","proceeding":null,"authors":["Chenjie Cao","Jingkai Zhou","Shikai Li","Jingyun Liang","Chaohui Yu","Fan Wang","xiangyang xue","Yanwei Fu"],"abstract":"Camera and human motion controls have been extensively studied for video generation, but existing approaches typically address them separately, suffering from limited data with high-quality annotations for both aspects. To overcome this, we present Uni3C, a unified 3D-enhanced framework for precise control of both camera and human motion in video generation. Uni3C includes two key contributions. First, we propose a plug-and-play control module trained with a frozen video generative backbone, PCDController, which utilizes unprojected point clouds from monocular depth to achieve accurate camera control. By leveraging the strong 3D priors of point clouds and the powerful capacities of video foundational models, PCDController shows impressive generalization, performing well regardless of whether the inference backbone is frozen or fine-tuned. This flexibility enables different modules of Uni3C to be trained in specific domains, i.e., either camera control or human motion control, reducing the dependency on jointly annotated data. Second, we propose a jointly aligned 3D world guidance for the inference phase that seamlessly integrates both scenic point clouds and SMPL-X characters to unify the control signals for camera and human motion, respectively. Extensive experiments confirm that PCDController enjoys strong robustness in driving camera motion for fine-tuned backbones of video generation. Uni3C substantially outperforms competitors in both camera controllability and human motion quality. Additionally, we collect tailored validation sets featuring challenging camera movements and human actions to validate the effectiveness of our method.","url_abs":"https://arxiv.org/abs/2504.14899v1","url_pdf":"https://arxiv.org/pdf/2504.14899v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uni3c-unifying-precisely-3d-enhanced-camera","repo_url":"https://github.com/ewrfcas/uni3c","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2504.14899","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.14899"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ewrfcas/uni3c","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"unverified":11},"by_repo_kind":{"official":{"samples":12,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9542161e9640b858","entry":"whitespace_clean","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/pipelines/pipeline_uni3c.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/pipelines/pipeline_uni3c.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9542161e9640b858"}},{"code_sha256_prefix":"8bc14e32eeec25b4","entry":"basic_clean","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/pipelines/pipeline_uni3c.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/pipelines/pipeline_uni3c.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8bc14e32eeec25b4"}},{"code_sha256_prefix":"ff47b803a9ac3fd5","entry":"batch_sample_rays","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/camera.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/camera.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff47b803a9ac3fd5"}},{"code_sha256_prefix":"7552f3526f085fdd","entry":"build_trajectory","repo":"ewrfcas/uni3c","repo_kind":"official","path":"alignment.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/alignment.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7552f3526f085fdd"}},{"code_sha256_prefix":"5d59a3eaa112ac1a","entry":"camera_center_normalization","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/camera.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/camera.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d59a3eaa112ac1a"}},{"code_sha256_prefix":"8bb483bf32c1034a","entry":"create_logger","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8bb483bf32c1034a"}},{"code_sha256_prefix":"4856df8ac84a6045","entry":"embed_rays","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/camera.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/camera.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4856df8ac84a6045"}},{"code_sha256_prefix":"26342ddf6119a713","entry":"get_boundaries_mask","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/pointcloud.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/pointcloud.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"26342ddf6119a713"}},{"code_sha256_prefix":"522fd30040cae1fc","entry":"load_video","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"522fd30040cae1fc"}},{"code_sha256_prefix":"105effd7cd6fb131","entry":"points_padding","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"105effd7cd6fb131"}},{"code_sha256_prefix":"bbacca188b4161f4","entry":"prompt_clean","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/pipelines/pipeline_uni3c.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/pipelines/pipeline_uni3c.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bbacca188b4161f4"}},{"code_sha256_prefix":"d9df3347334a07c9","entry":"shard_model","repo":"ewrfcas/uni3c","repo_kind":"official","path":"src/fsdp.py","file_url":"https://github.com/ewrfcas/uni3c/blob/HEAD/src/fsdp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9df3347334a07c9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}