Papers › Points to Patches: Enabling the Use of Self-Attention for 3D Shape Recognition

Points to Patches: Enabling the Use of Self-Attention for 3D Shape Recognition

8 Apr 2022arXiv:2204.03957archive 2025-07-28

Axel Berg, Magnus Oskarsson, Mark O'Connor

While the Transformer architecture has become ubiquitous in the machine learning field, its adaptation to 3D shape recognition is non-trivial. Due to its quadratic computational complexity, the self-attention operator quickly becomes inefficient as the set of input points grows larger. Furthermore, we find that the attention mechanism struggles to find useful connections between individual points on a global scale. In order to alleviate these problems, we propose a two-stage Point Transformer-in-Transformer (Point-TnT) approach which combines local and global attention mechanisms, enabling both individual points and patches of points to attend to each other effectively. Experiments on shape classification show that such an approach provides more useful features for downstream tasks than the baseline Transformer, while also being more computationally efficient. In addition, we also extend our method to feature matching for scene reconstruction, showing that it can be used in conjunction with existing scene reconstruction pipelines.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

axeber01/point-tnt officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

3D Feature Matching 3D Point Cloud Classification 3D Shape Recognition Point Cloud Registration

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
3D Point Cloud Classification	ModelNet40	Point-TnT	Number of params	3.9M	#86 of 111	Archive leaderboard	report
3D Point Cloud Classification	ModelNet40	Point-TnT	Overall Accuracy	92.6	#86 of 111	Archive leaderboard	report
3D Point Cloud Classification	ScanObjectNN	Point-TnT	FLOPs	1.19G	#63 of 77	Archive leaderboard	report
3D Point Cloud Classification	ScanObjectNN	Point-TnT	Mean Accuracy	81.0	#63 of 77	Archive leaderboard	report
3D Point Cloud Classification	ScanObjectNN	Point-TnT	Number of params	3.9M	#63 of 77	Archive leaderboard	report
3D Point Cloud Classification	ScanObjectNN	Point-TnT	Overall Accuracy	83.5	#63 of 77	Archive leaderboard	report
Point Cloud Registration	3DMatch Benchmark	DIP + Point-TnT	Feature Matching Recall	96.8	#6 of 15	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections