{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/samurai-shape-and-material-from-unconstrained","title":"SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections","arxiv_id":"2205.15768","date":"2022-05-31","proceeding":null,"authors":["Mark Boss","Andreas Engelhardt","Abhishek Kar","Yuanzhen Li","Deqing Sun","Jonathan T. Barron","Hendrik P. A. Lensch","Varun Jampani"],"abstract":"Inverse rendering of an object under entirely unknown capture conditions is a fundamental challenge in computer vision and graphics. Neural approaches such as NeRF have achieved photorealistic results on novel view synthesis, but they require known camera poses. Solving this problem with unknown camera poses is highly challenging as it requires joint optimization over shape, radiance, and pose. This problem is exacerbated when the input images are captured in the wild with varying backgrounds and illuminations. Standard pose estimation techniques fail in such image collections in the wild due to very few estimated correspondences across images. Furthermore, NeRF cannot relight a scene under any illumination, as it operates on radiance (the product of reflectance and illumination). We propose a joint optimization framework to estimate the shape, BRDF, and per-image camera pose and illumination. Our method works on in-the-wild online image collections of an object and produces relightable 3D assets for several use-cases such as AR/VR. To our knowledge, our method is the first to tackle this severely unconstrained task with minimal user interaction. Project page: https://markboss.me/publication/2022-samurai/ Video: https://youtu.be/LlYuGDjXp-8","url_abs":"https://arxiv.org/abs/2205.15768v1","url_pdf":"https://arxiv.org/pdf/2205.15768v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"samurai-shape-and-material-from-unconstrained","repo_url":"https://github.com/google/samurai","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"inverse-rendering","task_name":"Inverse Rendering"},{"task_slug":"nerf","task_name":"NeRF"},{"task_slug":"novel-view-synthesis","task_name":"Novel View Synthesis"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.15768","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.15768"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/samurai","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":12},"by_repo_kind":{"listed":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9f397292332658db","entry":"add_args","repo":"google/samurai","repo_kind":"listed","path":"train_samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/train_samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9f397292332658db"}},{"code_sha256_prefix":"d0f5cfa444c569da","entry":"add_args","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/config.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0f5cfa444c569da"}},{"code_sha256_prefix":"c3567310bee1cb8c","entry":"batch_cam_bce","repo":"google/samurai","repo_kind":"listed","path":"losses/samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/losses/samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3567310bee1cb8c"}},{"code_sha256_prefix":"9972e03030bf25dd","entry":"batch_cam_chi_squared_error","repo":"google/samurai","repo_kind":"listed","path":"losses/samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/losses/samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9972e03030bf25dd"}},{"code_sha256_prefix":"ec8c80f8d30fc57b","entry":"chi_squared_error","repo":"google/samurai","repo_kind":"listed","path":"losses/samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/losses/samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ec8c80f8d30fc57b"}},{"code_sha256_prefix":"1fda2c58600244de","entry":"get_targets_from_coordinates","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/input_generation_utils.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/input_generation_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1fda2c58600244de"}},{"code_sha256_prefix":"4d288518eb2cf074","entry":"prediction_to_specular","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/brdf_helper.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/brdf_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4d288518eb2cf074"}},{"code_sha256_prefix":"f5ab584c10bfbb2d","entry":"render_full_datapoint","repo":"google/samurai","repo_kind":"listed","path":"train_samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/train_samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f5ab584c10bfbb2d"}},{"code_sha256_prefix":"d73f64c830dc93d2","entry":"render_video","repo":"google/samurai","repo_kind":"listed","path":"train_samurai.py","file_url":"https://github.com/google/samurai/blob/HEAD/train_samurai.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d73f64c830dc93d2"}},{"code_sha256_prefix":"603ac1299e72e004","entry":"scale_diffuse","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/brdf_helper.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/brdf_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"603ac1299e72e004"}},{"code_sha256_prefix":"c55dd0b146f20dbe","entry":"scale_inputs","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/input_generation_utils.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/input_generation_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c55dd0b146f20dbe"}},{"code_sha256_prefix":"c25b3832e09b1acc","entry":"transform_specular_and_diffuse_prediction","repo":"google/samurai","repo_kind":"listed","path":"models/samurai/brdf_helper.py","file_url":"https://github.com/google/samurai/blob/HEAD/models/samurai/brdf_helper.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c25b3832e09b1acc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}