{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rpbg-towards-robust-neural-point-based","title":"RPBG: Towards Robust Neural Point-based Graphics in the Wild","arxiv_id":"2405.05663","date":"2024-05-09","proceeding":null,"authors":["Qingtian Zhu","Zizhuang Wei","Zhongtian Zheng","Yifan Zhan","Zhuyu Yao","Jiawang Zhang","Kejian Wu","Yinqiang Zheng"],"abstract":"Point-based representations have recently gained popularity in novel view synthesis, for their unique advantages, e.g., intuitive geometric representation, simple manipulation, and faster convergence. However, based on our observation, these point-based neural re-rendering methods are only expected to perform well under ideal conditions and suffer from noisy, patchy points and unbounded scenes, which are challenging to handle but defacto common in real applications. To this end, we revisit one such influential method, known as Neural Point-based Graphics (NPBG), as our baseline, and propose Robust Point-based Graphics (RPBG). We in-depth analyze the factors that prevent NPBG from achieving satisfactory renderings on generic datasets, and accordingly reform the pipeline to make it more robust to varying datasets in-the-wild. Inspired by the practices in image restoration, we greatly enhance the neural renderer to enable the attention-based correction of point visibility and the inpainting of incomplete rasterization, with only acceptable overheads. We also seek for a simple and lightweight alternative for environment modeling and an iterative method to alleviate the problem of poor geometry. By thorough evaluation on a wide range of datasets with different shooting conditions and camera trajectories, RPBG stably outperforms the baseline by a large margin, and exhibits its great robustness over state-of-the-art NeRF-based variants. Code available at https://github.com/QT-Zhu/RPBG.","url_abs":"https://arxiv.org/abs/2405.05663v2","url_pdf":"https://arxiv.org/pdf/2405.05663v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rpbg-towards-robust-neural-point-based","repo_url":"https://github.com/qt-zhu/rpbg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-restoration","task_name":"Image Restoration"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"}],"methods":[{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.05663","atlas_url":"https://app.syntology.ai/?focus=2405.05663","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.05663"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/QT-Zhu/RPBG","reach":null}],"summary":{"ran":8,"unverified":5},"by_repo_kind":{"official":{"samples":13,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"349e038d05b2cfce","entry":"AFF","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"349e038d05b2cfce"}},{"code_sha256_prefix":"14e81d2a2db05c54","entry":"DBlock","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14e81d2a2db05c54"}},{"code_sha256_prefix":"5642600dfdadb227","entry":"EBlock","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5642600dfdadb227"}},{"code_sha256_prefix":"4eb9ea5b3e878e63","entry":"FAM","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4eb9ea5b3e878e63"}},{"code_sha256_prefix":"60e29924690d6157","entry":"FourierUnit","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"60e29924690d6157"}},{"code_sha256_prefix":"24b0cd7a503871f5","entry":"GatedConv","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"24b0cd7a503871f5"}},{"code_sha256_prefix":"28239e93e175637f","entry":"ResBlock","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"28239e93e175637f"}},{"code_sha256_prefix":"2b1106c990cc59b9","entry":"SCM","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b1106c990cc59b9"}},{"code_sha256_prefix":"dde97bb42a868625","entry":"FFC","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dde97bb42a868625"}},{"code_sha256_prefix":"b1336d91e7c827d8","entry":"FFCResnetBlock","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1336d91e7c827d8"}},{"code_sha256_prefix":"1fdc8eec4ba8e86d","entry":"FFC_BN_ACT","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1fdc8eec4ba8e86d"}},{"code_sha256_prefix":"778754a205589fec","entry":"MIMOUNet","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"778754a205589fec"}},{"code_sha256_prefix":"65bcb1a744d547fe","entry":"SpectralTransform","repo":"QT-Zhu/RPBG","repo_kind":"official","path":"RPBG/models/mimounet.py","file_url":"https://github.com/QT-Zhu/RPBG/blob/HEAD/RPBG/models/mimounet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65bcb1a744d547fe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}