{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/streaming-4d-visual-geometry-transformer","title":"Streaming 4D Visual Geometry Transformer","arxiv_id":"2507.11539","date":"2025-07-15","proceeding":null,"authors":["Dong Zhuo","Wenzhao Zheng","Jiahe Guo","Yuqi Wu","Jie zhou","Jiwen Lu"],"abstract":"Perceiving and reconstructing 4D spatial-temporal geometry from videos is a fundamental yet challenging computer vision task. To facilitate interactive and real-time applications, we propose a streaming 4D visual geometry transformer that shares a similar philosophy with autoregressive large language models. We explore a simple and efficient design and employ a causal transformer architecture to process the input sequence in an online manner. We use temporal causal attention and cache the historical keys and values as implicit memory to enable efficient streaming long-term 4D reconstruction. This design can handle real-time 4D reconstruction by incrementally integrating historical information while maintaining high-quality spatial consistency. For efficient training, we propose to distill knowledge from the dense bidirectional visual geometry grounded transformer (VGGT) to our causal model. For inference, our model supports the migration of optimized efficient attention operator (e.g., FlashAttention) from the field of large language models. Extensive experiments on various 4D geometry perception benchmarks demonstrate that our model increases the inference speed in online scenarios while maintaining competitive performance, paving the way for scalable and interactive 4D vision systems. Code is available at: https://github.com/wzzheng/StreamVGGT.","url_abs":"https://arxiv.org/abs/2507.11539v1","url_pdf":"https://arxiv.org/pdf/2507.11539v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"streaming-4d-visual-geometry-transformer","repo_url":"https://github.com/wzzheng/streamvggt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"4d-reconstruction","task_name":"4D reconstruction"},{"task_slug":"philosophy","task_name":"Philosophy"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2507.11539","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2507.11539"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wzzheng/streamvggt","reach":{"status":"ok","spdx":"NOASSERTION"}}],"summary":{"ran_fixture":2,"ran_honours":2,"unverified":9},"by_repo_kind":{"official":{"samples":13,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":13,"samples":[{"code_sha256_prefix":"b881c649136fa014","entry":"base_pose_act","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/head_act.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/head_act.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b881c649136fa014"}},{"code_sha256_prefix":"c157f5b112b3a392","entry":"drop_path","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/layers/drop_path.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/layers/drop_path.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c157f5b112b3a392"}},{"code_sha256_prefix":"3c04490a215299f2","entry":"make_2tuple","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/layers/patch_embed.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/layers/patch_embed.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c04490a215299f2"}},{"code_sha256_prefix":"ed311bf19f15387b","entry":"modulate","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/camera_head.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/camera_head.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed311bf19f15387b"}},{"code_sha256_prefix":"6e3d1e3fa82b57d9","entry":"activate_head","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/head_act.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/head_act.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"6e3d1e3fa82b57d9"}},{"code_sha256_prefix":"732f3a3c6b94593c","entry":"activate_pose","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/head_act.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/head_act.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"732f3a3c6b94593c"}},{"code_sha256_prefix":"5d16d4d4fc573ac7","entry":"add_residual","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/layers/block.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/layers/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d16d4d4fc573ac7"}},{"code_sha256_prefix":"ec669c1b58089748","entry":"create_uv_grid","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/utils.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"ec669c1b58089748"}},{"code_sha256_prefix":"f43a4f443063d830","entry":"custom_interpolate","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/dpt_head.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/dpt_head.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f43a4f443063d830"}},{"code_sha256_prefix":"a2ec87bca0ee4781","entry":"drop_add_residual_stochastic_depth","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/layers/block.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/layers/block.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"a2ec87bca0ee4781"}},{"code_sha256_prefix":"5be3610fa1fee19e","entry":"get_branges_scales","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/layers/block.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/layers/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5be3610fa1fee19e"}},{"code_sha256_prefix":"731bd479ffcd13a9","entry":"make_sincos_pos_embed","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/utils.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"731bd479ffcd13a9"}},{"code_sha256_prefix":"910e268f54177e2b","entry":"position_grid_to_embed","repo":"wzzheng/streamvggt","repo_kind":"official","path":"src/streamvggt/heads/utils.py","file_url":"https://github.com/wzzheng/streamvggt/blob/HEAD/src/streamvggt/heads/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"910e268f54177e2b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}