{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bpnp-further-empowering-end-to-end-learning","title":"End-to-End Learnable Geometric Vision by Backpropagating PnP Optimization","arxiv_id":"1909.06043","date":"2019-09-13","proceeding":"CVPR 2020 6","authors":["Bo Chen","Alvaro Parra","Jiewei Cao","Nan Li","Tat-Jun Chin"],"abstract":"Deep networks excel in learning patterns from large amounts of data. On the other hand, many geometric vision tasks are specified as optimization problems. To seamlessly combine deep learning and geometric vision, it is vital to perform learning and geometric optimization end-to-end. Towards this aim, we present BPnP, a novel network module that backpropagates gradients through a Perspective-n-Points (PnP) solver to guide parameter updates of a neural network. Based on implicit differentiation, we show that the gradients of a \"self-contained\" PnP solver can be derived accurately and efficiently, as if the optimizer block were a differentiable function. We validate BPnP by incorporating it in a deep model that can learn camera intrinsics, camera extrinsics (poses) and 3D structure from training datasets. Further, we develop an end-to-end trainable pipeline for object pose estimation, which achieves greater accuracy by combining feature-based heatmap losses with 2D-3D reprojection errors. Since our approach can be extended to other optimization problems, our work helps to pave the way to perform learnable geometric vision in a principled manner. Our PyTorch implementation of BPnP is available on http://github.com/BoChenYS/BPnP.","url_abs":"https://arxiv.org/abs/1909.06043v3","url_pdf":"https://arxiv.org/pdf/1909.06043v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bpnp-further-empowering-end-to-end-learning","repo_url":"https://github.com/BoChenYS/BPnP","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"bpnp-further-empowering-end-to-end-learning","repo_url":"https://github.com/tzole1155/EndToEndObjectPose","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"6d-pose-estimation-1","task_name":"6D Pose Estimation"},{"task_slug":"6d-pose-estimation","task_name":"6D Pose Estimation using RGB"}],"methods":[{"method_slug":"heatmap","method_name":"Heatmap"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/6d-pose-estimation-on-linemod","task":"6D Pose Estimation using RGB","dataset":"LineMOD","model":"HRNet (DSNT + BPnP)","rank_in_archive_order":9,"of":22,"metrics":{"Mean ADD":"93.3"},"uses_additional_data":false},{"leaderboard":"/sota/6d-pose-estimation-on-linemod","task":"6D Pose Estimation using RGB","dataset":"LineMOD","model":"HRNet+DSNT+BPnP","rank_in_archive_order":22,"of":22,"metrics":{"Accuracy":"99.21%","Accuracy (ADD)":"93.27%"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1909.06043","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.06043"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/BoChenYS/BPnP","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tzole1155/EndToEndObjectPose","reach":{"status":"ok"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d35e8d21d1b9d980","entry":"batch_project","repo":"BoChenYS/BPnP","repo_kind":"official","path":"BPnP.py","file_url":"https://github.com/BoChenYS/BPnP/blob/HEAD/BPnP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d35e8d21d1b9d980"}},{"code_sha256_prefix":"58b72ebb953d3283","entry":"get_coefs","repo":"BoChenYS/BPnP","repo_kind":"official","path":"BPnP.py","file_url":"https://github.com/BoChenYS/BPnP/blob/HEAD/BPnP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58b72ebb953d3283"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}