{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/im4d-high-fidelity-and-real-time-novel-view","title":"Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes","arxiv_id":"2310.08585","date":"2023-10-12","proceeding":null,"authors":["Haotong Lin","Sida Peng","Zhen Xu","Tao Xie","Xingyi He","Hujun Bao","Xiaowei Zhou"],"abstract":"This paper aims to tackle the challenge of dynamic view synthesis from multi-view videos. The key observation is that while previous grid-based methods offer consistent rendering, they fall short in capturing appearance details of a complex dynamic scene, a domain where multi-view image-based rendering methods demonstrate the opposite properties. To combine the best of two worlds, we introduce Im4D, a hybrid scene representation that consists of a grid-based geometry representation and a multi-view image-based appearance representation. Specifically, the dynamic geometry is encoded as a 4D density function composed of spatiotemporal feature planes and a small MLP network, which globally models the scene structure and facilitates the rendering consistency. We represent the scene appearance by the original multi-view videos and a network that learns to predict the color of a 3D point from image features, instead of memorizing detailed appearance totally with networks, thereby naturally making the learning of networks easier. Our method is evaluated on five dynamic view synthesis datasets including DyNeRF, ZJU-MoCap, NHR, DNA-Rendering and ENeRF-Outdoor datasets. The results show that Im4D exhibits state-of-the-art performance in rendering quality and can be trained efficiently, while realizing real-time rendering with a speed of 79.8 FPS for 512x512 images, on a single RTX 3090 GPU.","url_abs":"https://arxiv.org/abs/2310.08585v1","url_pdf":"https://arxiv.org/pdf/2310.08585v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"im4d-high-fidelity-and-real-time-novel-view","repo_url":"https://github.com/zju3dv/im4d","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.08585","atlas_url":"https://app.syntology.ai/?focus=2310.08585","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.08585"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zju3dv/im4d","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"bc1bedb7bee102cf","entry":"list_to_str","repo":"zju3dv/im4d","repo_kind":"official","path":"lib/config/config.py","file_url":"https://github.com/zju3dv/im4d/blob/HEAD/lib/config/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"bc1bedb7bee102cf"}},{"code_sha256_prefix":"411a24e38dae9a0d","entry":"load_yaml_file","repo":"zju3dv/im4d","repo_kind":"official","path":"lib/config/config.py","file_url":"https://github.com/zju3dv/im4d/blob/HEAD/lib/config/config.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"411a24e38dae9a0d"}},{"code_sha256_prefix":"87df4437d34bb35f","entry":"make_data_sampler","repo":"zju3dv/im4d","repo_kind":"official","path":"lib/datasets/make_dataset.py","file_url":"https://github.com/zju3dv/im4d/blob/HEAD/lib/datasets/make_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87df4437d34bb35f"}},{"code_sha256_prefix":"de8a4d6a5ad9880c","entry":"none_collate","repo":"zju3dv/im4d","repo_kind":"official","path":"lib/datasets/collate_batch.py","file_url":"https://github.com/zju3dv/im4d/blob/HEAD/lib/datasets/collate_batch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"de8a4d6a5ad9880c"}},{"code_sha256_prefix":"65d6513bc44b700c","entry":"load_cfg","repo":"zju3dv/im4d","repo_kind":"official","path":"lib/config/yacs.py","file_url":"https://github.com/zju3dv/im4d/blob/HEAD/lib/config/yacs.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65d6513bc44b700c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}