{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scene-aware-egocentric-3d-human-pose","title":"Scene-aware Egocentric 3D Human Pose Estimation","arxiv_id":"2212.11684","date":"2022-12-20","proceeding":"CVPR 2023 1","authors":["Jian Wang","Lingjie Liu","Weipeng Xu","Kripasindhu Sarkar","Diogo Luvizon","Christian Theobalt"],"abstract":"Egocentric 3D human pose estimation with a single head-mounted fisheye camera has recently attracted attention due to its numerous applications in virtual and augmented reality. Existing methods still struggle in challenging poses where the human body is highly occluded or is closely interacting with the scene. To address this issue, we propose a scene-aware egocentric pose estimation method that guides the prediction of the egocentric pose with scene constraints. To this end, we propose an egocentric depth estimation network to predict the scene depth map from a wide-view egocentric fisheye camera while mitigating the occlusion of the human body with a depth-inpainting network. Next, we propose a scene-aware pose estimation network that projects the 2D image features and estimated depth map of the scene into a voxel space and regresses the 3D pose with a V2V network. The voxel-based feature representation provides the direct geometric connection between 2D image features and scene geometry, and further facilitates the V2V network to constrain the predicted pose based on the estimated scene geometry. To enable the training of the aforementioned networks, we also generated a synthetic dataset, called EgoGTA, and an in-the-wild dataset based on EgoPW, called EgoPW-Scene. The experimental results of our new evaluation sequences show that the predicted 3D egocentric poses are accurate and physically plausible in terms of human-scene interaction, demonstrating that our method outperforms the state-of-the-art methods both quantitatively and qualitatively.","url_abs":"https://arxiv.org/abs/2212.11684v3","url_pdf":"https://arxiv.org/pdf/2212.11684v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scene-aware-egocentric-3d-human-pose","repo_url":"https://github.com/jianwang-mpi/SceneEgo","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-human-pose-estimation","task_name":"3D Human Pose Estimation"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"egocentric-pose-estimation","task_name":"Egocentric Pose Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"}],"methods":[],"datasets_introduced":[{"slug":"egopw-scene","name":"EgoPW-Scene","full_name":""},{"slug":"sceneego","name":"SceneEgo","full_name":"SceneEgo Test Dataset"}],"methods_introduced":[],"results":[{"leaderboard":"/sota/egocentric-pose-estimation-on-globalegomocap","task":"Egocentric Pose Estimation","dataset":"GlobalEgoMocap Test Dataset","model":"SceneEgo","rank_in_archive_order":3,"of":7,"metrics":{"Average MPJPE (mm)":"76.50","PA-MPJPE":"61.92"},"uses_additional_data":true},{"leaderboard":"/sota/egocentric-pose-estimation-on-sceneego","task":"Egocentric Pose Estimation","dataset":"SceneEgo","model":"SceneEgo","rank_in_archive_order":4,"of":8,"metrics":{"Average MPJPE (mm)":"118.5","PA-MPJPE":"92.75"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.11684","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.11684"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jianwang-mpi/SceneEgo","reach":null}],"summary":{"ran_draft_wrong":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"fac5364e2f53c6db","entry":"conv3x3","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fac5364e2f53c6db"}},{"code_sha256_prefix":"55c58ef5f795e3d1","entry":"get_max_preds","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"55c58ef5f795e3d1"}},{"code_sha256_prefix":"bb7cf362aae536cb","entry":"load_state_dict","repo":"jianwang-mpi/SceneEgo","repo_kind":"official","path":"network/pose_resnet.py","file_url":"https://github.com/jianwang-mpi/SceneEgo/blob/HEAD/network/pose_resnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bb7cf362aae536cb"}},{"code_sha256_prefix":"f7ea42152a342da0","entry":"get_pose_net","repo":"jianwang-mpi/SceneEgo","repo_kind":"official","path":"network/pose_resnet.py","file_url":"https://github.com/jianwang-mpi/SceneEgo/blob/HEAD/network/pose_resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f7ea42152a342da0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}