{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/3d-aware-scene-manipulation-via-inverse","title":"3D-Aware Scene Manipulation via Inverse Graphics","arxiv_id":"1808.09351","date":"2018-08-28","proceeding":"NeurIPS 2018 12","authors":["Shunyu Yao","Tzu Ming Harry Hsu","Jun-Yan Zhu","Jiajun Wu","Antonio Torralba","William T. Freeman","Joshua B. Tenenbaum"],"abstract":"We aim to obtain an interpretable, expressive, and disentangled scene\nrepresentation that contains comprehensive structural and textural information\nfor each object. Previous scene representations learned by neural networks are\noften uninterpretable, limited to a single object, or lacking 3D knowledge. In\nthis work, we propose 3D scene de-rendering networks (3D-SDN) to address the\nabove issues by integrating disentangled representations for semantics,\ngeometry, and appearance into a deep generative model. Our scene encoder\nperforms inverse graphics, translating a scene into a structured object-wise\nrepresentation. Our decoder has two components: a differentiable shape renderer\nand a neural texture generator. The disentanglement of semantics, geometry, and\nappearance supports 3D-aware scene manipulation, e.g., rotating and moving\nobjects freely while keeping the consistent shape and texture, and changing the\nobject appearance without affecting its shape. Experiments demonstrate that our\nediting scheme based on 3D-SDN is superior to its 2D counterpart.","url_abs":"http://arxiv.org/abs/1808.09351v4","url_pdf":"http://arxiv.org/pdf/1808.09351v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"3d-aware-scene-manipulation-via-inverse","repo_url":"https://github.com/ysymyth/3D-SDN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"object","task_name":"Object"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.09351","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}