{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/im2avatar-colorful-3d-reconstruction-from-a","title":"Im2Avatar: Colorful 3D Reconstruction from a Single Image","arxiv_id":"1804.06375","date":"2018-04-17","proceeding":null,"authors":["Yongbin Sun","Ziwei Liu","Yue Wang","Sanjay E. Sarma"],"abstract":"Existing works on single-image 3D reconstruction mainly focus on shape\nrecovery. In this work, we study a new problem, that is, simultaneously\nrecovering 3D shape and surface color from a single image, namely \"colorful 3D\nreconstruction\". This problem is both challenging and intriguing because the\nability to infer textured 3D model from a single image is at the core of visual\nunderstanding. Here, we propose an end-to-end trainable framework, Colorful\nVoxel Network (CVN), to tackle this problem. Conditioned on a single 2D input,\nCVN learns to decompose shape and surface color information of a 3D object into\na 3D shape branch and a surface color branch, respectively. Specifically, for\nthe shape recovery, we generate a shape volume with the state of its voxels\nindicating occupancy. For the surface color recovery, we combine the strength\nof appearance hallucination and geometric projection by concurrently learning a\nregressed color volume and a 2D-to-3D flow volume, which are then fused into a\nblended color volume. The final textured 3D model is obtained by sampling color\nfrom the blended color volume at the positions of occupied voxels in the shape\nvolume. To handle the severe sparse volume representations, a novel loss\nfunction, Mean Squared False Cross-Entropy Loss (MSFCEL), is designed.\nExtensive experiments demonstrate that our approach achieves significant\nimprovement over baselines, and shows great generalization across diverse\nobject categories and arbitrary viewpoints.","url_abs":"http://arxiv.org/abs/1804.06375v1","url_pdf":"http://arxiv.org/pdf/1804.06375v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"im2avatar-colorful-3d-reconstruction-from-a","repo_url":"https://github.com/syb7573330/im2avatar","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"hallucination","task_name":"Hallucination"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.06375","atlas_url":"https://app.syntology.ai/?focus=1804.06375","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}