{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gsnet-joint-vehicle-pose-and-shape","title":"GSNet: Joint Vehicle Pose and Shape Reconstruction with Geometrical and Scene-aware Supervision","arxiv_id":"2007.13124","date":"2020-07-26","proceeding":"ECCV 2020 8","authors":["Lei Ke","Shichao Li","Yanan sun","Yu-Wing Tai","Chi-Keung Tang"],"abstract":"We present a novel end-to-end framework named as GSNet (Geometric and Scene-aware Network), which jointly estimates 6DoF poses and reconstructs detailed 3D car shapes from single urban street view. GSNet utilizes a unique four-way feature extraction and fusion scheme and directly regresses 6DoF poses and shapes in a single forward pass. Extensive experiments show that our diverse feature extraction and fusion scheme can greatly improve model performance. Based on a divide-and-conquer 3D shape representation strategy, GSNet reconstructs 3D vehicle shape with great detail (1352 vertices and 2700 faces). This dense mesh representation further leads us to consider geometrical consistency and scene context, and inspires a new multi-objective loss function to regularize network training, which in turn improves the accuracy of 6D pose estimation and validates the merit of jointly performing both tasks. We evaluate GSNet on the largest multi-task ApolloCar3D benchmark and achieve state-of-the-art performance both quantitatively and qualitatively. Project page is available at https://lkeab.github.io/gsnet/.","url_abs":"https://arxiv.org/abs/2007.13124v1","url_pdf":"https://arxiv.org/pdf/2007.13124v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gsnet-joint-vehicle-pose-and-shape","repo_url":"https://github.com/lkeab/gsnet","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-car-instance-understanding","task_name":"3D Car Instance Understanding"},{"task_slug":"3d-pose-estimation","task_name":"3D Pose Estimation"},{"task_slug":"3d-reconstruction","task_name":"3D Reconstruction"},{"task_slug":"3d-shape-modeling","task_name":"3D Shape Modeling"},{"task_slug":"3d-shape-reconstruction","task_name":"3D Shape Reconstruction"},{"task_slug":"3d-shape-reconstruction-from-a-single-2d","task_name":"3D Shape Reconstruction From A Single 2D Image"},{"task_slug":"3d-shape-representation","task_name":"3D Shape Representation"},{"task_slug":"6d-pose-estimation-1","task_name":"6D Pose Estimation"},{"task_slug":"6d-pose-estimation","task_name":"6D Pose Estimation using RGB"},{"task_slug":"autonomous-driving","task_name":"Autonomous Driving"},{"task_slug":"autonomous-vehicles","task_name":"Autonomous Vehicles"},{"task_slug":"keypoint-detection","task_name":"Keypoint Detection"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"self-driving-cars","task_name":"Self-Driving Cars"},{"task_slug":"vehicle-key-point-and-orientation-estimation","task_name":"Vehicle Key-Point and Orientation Estimation"},{"task_slug":"vehicle-pose-estimation","task_name":"Vehicle Pose Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-car-instance-understanding-on-apollocar3d","task":"3D Car Instance Understanding","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":2,"of":2,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/3d-pose-estimation-on-apollocar3d","task":"3D Pose Estimation","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/3d-reconstruction-on-apollocar3d","task":"3D Reconstruction","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/3d-shape-reconstruction-on-apollocar3d","task":"3D Shape Reconstruction","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/3d-shape-reconstruction-from-a-single-2d","task":"3D Shape Reconstruction From A Single 2D Image","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":true},{"leaderboard":"/sota/6d-pose-estimation-on-apollocar3d","task":"6D Pose Estimation","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/6d-pose-estimation-using-rgb-on-apollocar3d","task":"6D Pose Estimation using RGB","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/autonomous-driving-on-apollocar3d","task":"Autonomous Driving","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/autonomous-vehicles-on-apollocar3d","task":"Autonomous Vehicles","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":2,"of":2,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/keypoint-detection-on-apollocar3d","task":"Keypoint Detection","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/pose-estimation-on-apollocar3d","task":"Pose Estimation","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/vehicle-key-point-and-orientation-estimation","task":"Vehicle Key-Point and Orientation Estimation","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false},{"leaderboard":"/sota/vehicle-pose-estimation-on-apollocar3d","task":"Vehicle Pose Estimation","dataset":"ApolloCar3D","model":"GSNet","rank_in_archive_order":1,"of":1,"metrics":{"A3DP":"20.21"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2007.13124","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}