Papers › Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction

Geometry-Biased Transformer for Robust Multi-View 3D Human Pose Reconstruction

28 Dec 2023arXiv:2312.17106archive 2025-07-28

Olivier Moliner, Sangxia Huang, Kalle Åström

We address the challenges in estimating 3D human poses from multiple views under occlusion and with limited overlapping views. We approach multi-view, single-person 3D human pose reconstruction as a regression problem and propose a novel encoder-decoder Transformer architecture to estimate 3D poses from multi-view 2D pose sequences. The encoder refines 2D skeleton joints detected across different views and times, fusing multi-view and temporal information through global self-attention. We enhance the encoder by incorporating a geometry-biased attention mechanism, effectively leveraging geometric relationships between views. Additionally, we use detection scores provided by the 2D pose detector to further guide the encoder's attention based on the reliability of the 2D detections. The decoder subsequently regresses the 3D pose sequence from these refined tokens, using pre-defined queries for each joint. To enhance the generalization of our method to unseen scenes and improve resilience to missing joints, we implement strategies including scene centering, synthetic views, and token dropout. We conduct extensive experiments on three benchmark public datasets, Human3.6M, CMU Panoptic and Occlusion-Persons. Our results demonstrate the efficacy of our approach, particularly in occluded scenes and when few views are available, which are traditionally challenging scenarios for triangulation-based methods.

Paper PDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Human Pose Estimation 3D Multi-Person Pose Estimation Decoder

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
3D Human Pose Estimation	Human3.6M	Geometry-Biased Transformer (HRNet)	Average MPJPE (mm)	26.0	#2 of 88	Archive leaderboard	report
3D Human Pose Estimation	Human3.6M	Geometry-Biased Transformer (HRNet)	Multi-View or Monocular	Multi-View	#2 of 88	Archive leaderboard	report
3D Human Pose Estimation	Human3.6M	Geometry-Biased Transformer (HRNet)	Using 2D ground-truth joints	No	#2 of 88	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections