{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/raynet-learning-volumetric-3d-reconstruction","title":"RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials","arxiv_id":"1901.01535","date":"2019-01-06","proceeding":"CVPR 2018 6","authors":["Despoina Paschalidou","Ali Osman Ulusoy","Carolin Schmitt","Luc van Gool","Andreas Geiger"],"abstract":"In this paper, we consider the problem of reconstructing a dense 3D model\nusing images captured from different views. Recent methods based on\nconvolutional neural networks (CNN) allow learning the entire task from data.\nHowever, they do not incorporate the physics of image formation such as\nperspective geometry and occlusion. Instead, classical approaches based on\nMarkov Random Fields (MRF) with ray-potentials explicitly model these physical\nprocesses, but they cannot cope with large surface appearance variations across\ndifferent viewpoints. In this paper, we propose RayNet, which combines the\nstrengths of both frameworks. RayNet integrates a CNN that learns\nview-invariant feature representations with an MRF that explicitly encodes the\nphysics of perspective projection and occlusion. We train RayNet end-to-end\nusing empirical risk minimization. We thoroughly evaluate our approach on\nchallenging real-world datasets and demonstrate its benefits over a piece-wise\ntrained baseline, hand-crafted models as well as other learning-based\napproaches.","url_abs":"http://arxiv.org/abs/1901.01535v1","url_pdf":"http://arxiv.org/pdf/1901.01535v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"raynet-learning-volumetric-3d-reconstruction","repo_url":"https://github.com/paschalidoud/raynet","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1901.01535","atlas_url":"https://app.syntology.ai/?focus=1901.01535","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}