{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/framenet-learning-local-canonical-frames-of","title":"FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image","arxiv_id":"1903.12305","date":"2019-03-29","proceeding":"ICCV 2019 10","authors":["Jingwei Huang","Yichao Zhou","Thomas Funkhouser","Leonidas Guibas"],"abstract":"In this work, we introduce the novel problem of identifying dense canonical\n3D coordinate frames from a single RGB image. We observe that each pixel in an\nimage corresponds to a surface in the underlying 3D geometry, where a canonical\nframe can be identified as represented by three orthogonal axes, one along its\nnormal direction and two in its tangent plane. We propose an algorithm to\npredict these axes from RGB. Our first insight is that canonical frames\ncomputed automatically with recently introduced direction field synthesis\nmethods can provide training data for the task. Our second insight is that\nnetworks designed for surface normal prediction provide better results when\ntrained jointly to predict canonical frames, and even better when trained to\nalso predict 2D projections of canonical frames. We conjecture this is because\nprojections of canonical tangent directions often align with local gradients in\nimages, and because those directions are tightly linked to 3D canonical frames\nthrough projective geometry and orthogonality constraints. In our experiments,\nwe find that our method predicts 3D canonical frames that can be used in\napplications ranging from surface normal estimation, feature matching, and\naugmented reality.","url_abs":"http://arxiv.org/abs/1903.12305v1","url_pdf":"http://arxiv.org/pdf/1903.12305v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"framenet-learning-local-canonical-frames-of","repo_url":"https://github.com/hjwdzh/FrameNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-geometry","task_name":"3D geometry"},{"task_slug":"surface-normal-estimation","task_name":"Surface Normal Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1903.12305","atlas_url":"https://app.syntology.ai/?focus=1903.12305","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}