{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/orv-4d-occupancy-centric-robot-video","title":"ORV: 4D Occupancy-centric Robot Video Generation","arxiv_id":"2506.03079","date":"2025-06-03","proceeding":null,"authors":["Xiuyu Yang","Bohan Li","Shaocong Xu","Nan Wang","Chongjie Ye","Zhaoxi Chen","Minghan Qin","Yikang Ding","Xin Jin","Hang Zhao","Hao Zhao"],"abstract":"Acquiring real-world robotic simulation data through teleoperation is notoriously time-consuming and labor-intensive. Recently, action-driven generative models have gained widespread adoption in robot learning and simulation, as they eliminate safety concerns and reduce maintenance efforts. However, the action sequences used in these methods often result in limited control precision and poor generalization due to their globally coarse alignment. To address these limitations, we propose ORV, an Occupancy-centric Robot Video generation framework, which utilizes 4D semantic occupancy sequences as a fine-grained representation to provide more accurate semantic and geometric guidance for video generation. By leveraging occupancy-based representations, ORV enables seamless translation of simulation data into photorealistic robot videos, while ensuring high temporal consistency and precise controllability. Furthermore, our framework supports the simultaneous generation of multi-view videos of robot gripping operations - an important capability for downstream robotic learning tasks. Extensive experimental results demonstrate that ORV consistently outperforms existing baseline methods across various datasets and sub-tasks. Demo, Code and Model: https://orangesodahub.github.io/ORV","url_abs":"https://arxiv.org/abs/2506.03079v1","url_pdf":"https://arxiv.org/pdf/2506.03079v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"orv-4d-occupancy-centric-robot-video","repo_url":"https://github.com/orangesodahub/orv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2506.03079","atlas_url":"https://app.syntology.ai/?focus=2506.03079","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.03079"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/orangesodahub/orv","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_fixture":1,"ran_draft_wrong":1,"unverified":14},"by_repo_kind":{"official":{"samples":18,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5fb220ab292f1b53","entry":"alpha2rotm","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5fb220ab292f1b53"}},{"code_sha256_prefix":"73992eb6c4ba4d3a","entry":"beta2rotm","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"73992eb6c4ba4d3a"}},{"code_sha256_prefix":"734fd43f6aaef115","entry":"get_resize_crop_region_for_grid","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/utils.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"734fd43f6aaef115"}},{"code_sha256_prefix":"129b804760b3115f","entry":"zero_module","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/models/components.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/models/components.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"129b804760b3115f"}},{"code_sha256_prefix":"426f0f1dab708c51","entry":"apply_depth_colormap","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/gs_render.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"426f0f1dab708c51"}},{"code_sha256_prefix":"35532b2f52e36b95","entry":"apply_semantic_colormap","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/gs_render.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35532b2f52e36b95"}},{"code_sha256_prefix":"7295a28acba3b4d6","entry":"check_frames","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/encode_dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/encode_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7295a28acba3b4d6"}},{"code_sha256_prefix":"e7ebace65f353a0c","entry":"check_height","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/encode_dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/encode_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7ebace65f353a0c"}},{"code_sha256_prefix":"31be697a051d0a8f","entry":"check_width","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/encode_dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/encode_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"31be697a051d0a8f"}},{"code_sha256_prefix":"e9a26f1eb44c56f7","entry":"compute_prompt_embeddings","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/models/text_encoder.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/models/text_encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e9a26f1eb44c56f7"}},{"code_sha256_prefix":"a1f748bf48f340a2","entry":"create_full_center_coords","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/gs_render.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/gs_render.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1f748bf48f340a2"}},{"code_sha256_prefix":"431f6ab9b15dad26","entry":"depths_to_points","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/prepare_dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/prepare_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"431f6ab9b15dad26"}},{"code_sha256_prefix":"59483a2ba1d88b48","entry":"encode_prompt","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/models/text_encoder.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/models/text_encoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59483a2ba1d88b48"}},{"code_sha256_prefix":"fde60ae068576551","entry":"gamma2rotm","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fde60ae068576551"}},{"code_sha256_prefix":"bed5d72fa03734e8","entry":"get_gradient_norm","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/utils.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bed5d72fa03734e8"}},{"code_sha256_prefix":"c8383b7f8ebf0d27","entry":"get_optimizer","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/utils.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8383b7f8ebf0d27"}},{"code_sha256_prefix":"67bd054a96fb4f6f","entry":"is_valid_image_imagelist","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/models/components.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/models/components.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67bd054a96fb4f6f"}},{"code_sha256_prefix":"47bf3c7cfcece953","entry":"points_to_voxels","repo":"orangesodahub/orv","repo_kind":"official","path":"orv/dataset/prepare_dataset.py","file_url":"https://github.com/orangesodahub/orv/blob/HEAD/orv/dataset/prepare_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"47bf3c7cfcece953"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}