{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/semantic-ray-learning-a-generalizable","title":"Semantic Ray: Learning a Generalizable Semantic Field with Cross-Reprojection Attention","arxiv_id":"2303.13014","date":"2023-03-23","proceeding":"CVPR 2023 1","authors":["Fangfu Liu","Chubin Zhang","Yu Zheng","Yueqi Duan"],"abstract":"In this paper, we aim to learn a semantic radiance field from multiple scenes that is accurate, efficient and generalizable. While most existing NeRFs target at the tasks of neural scene rendering, image synthesis and multi-view reconstruction, there are a few attempts such as Semantic-NeRF that explore to learn high-level semantic understanding with the NeRF structure. However, Semantic-NeRF simultaneously learns color and semantic label from a single ray with multiple heads, where the single ray fails to provide rich semantic information. As a result, Semantic NeRF relies on positional encoding and needs to train one specific model for each scene. To address this, we propose Semantic Ray (S-Ray) to fully exploit semantic information along the ray direction from its multi-view reprojections. As directly performing dense attention over multi-view reprojected rays would suffer from heavy computational cost, we design a Cross-Reprojection Attention module with consecutive intra-view radial and cross-view sparse attentions, which decomposes contextual information along reprojected rays and cross multiple views and then collects dense connections by stacking the modules. Experiments show that our S-Ray is able to learn from multiple scenes, and it presents strong generalization ability to adapt to unseen scenes.","url_abs":"https://arxiv.org/abs/2303.13014v1","url_pdf":"https://arxiv.org/pdf/2303.13014v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"semantic-ray-learning-a-generalizable","repo_url":"https://github.com/liuff19/Semantic-Ray","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"generalizable-novel-view-synthesis","task_name":"Generalizable Novel View Synthesis"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"dense-connections","method_name":"Dense Connections"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.13014","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.13014"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liuff19/Semantic-Ray","reach":null}],"summary":{"ran":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9efc375428ecb470","entry":"CRANet","repo":"liuff19/Semantic-Ray","repo_kind":"official","path":"network/cranet.py","file_url":"https://github.com/liuff19/Semantic-Ray/blob/HEAD/network/cranet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9efc375428ecb470"}},{"code_sha256_prefix":"c74f7a0c0877115b","entry":"IBRNetWithNeuRay","repo":"liuff19/Semantic-Ray","repo_kind":"official","path":"network/cranet.py","file_url":"https://github.com/liuff19/Semantic-Ray/blob/HEAD/network/cranet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c74f7a0c0877115b"}},{"code_sha256_prefix":"4b49c3776169984f","entry":"MultiHeadAttention","repo":"liuff19/Semantic-Ray","repo_kind":"official","path":"network/cranet.py","file_url":"https://github.com/liuff19/Semantic-Ray/blob/HEAD/network/cranet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4b49c3776169984f"}},{"code_sha256_prefix":"3e26762debce1347","entry":"ScaledDotProductAttention","repo":"liuff19/Semantic-Ray","repo_kind":"official","path":"network/cranet.py","file_url":"https://github.com/liuff19/Semantic-Ray/blob/HEAD/network/cranet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e26762debce1347"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}