{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/positional-prompt-tuning-for-efficient-3d","title":"Positional Prompt Tuning for Efficient 3D Representation Learning","arxiv_id":"2408.11567","date":"2024-08-21","proceeding":null,"authors":["Shaochen Zhang","Zekun Qi","Runpei Dong","Xiuxiu Bai","Xing Wei"],"abstract":"Point cloud analysis has achieved significant development and is well-performed in multiple downstream tasks like point cloud classification and segmentation, etc. Being conscious of the simplicity of the position encoding structure in Transformer-based architectures, we attach importance to the position encoding as a high-dimensional part and the patch encoder to offer multi-scale information. Together with the sequential Transformer, the whole module with position encoding comprehensively constructs a multi-scale feature abstraction module that considers both the local parts from the patch and the global parts from center points as position encoding. With only a few parameters, the position embedding module fits the setting of PEFT (Parameter-Efficient Fine-Tuning) tasks pretty well. Thus we unfreeze these parameters as a fine-tuning part. At the same time, we review the existing prompt and adapter tuning methods, proposing a fresh way of prompts and synthesizing them with adapters as dynamic adjustments. Our Proposed method of PEFT tasks, namely PPT, with only 1.05% of parameters for training, gets state-of-the-art results in several mainstream datasets, such as 95.01% accuracy in the ScanObjectNN OBJ_BG dataset. Codes will be released at https://github.com/zsc000722/PPT.","url_abs":"https://arxiv.org/abs/2408.11567v1","url_pdf":"https://arxiv.org/pdf/2408.11567v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"positional-prompt-tuning-for-efficient-3d","repo_url":"https://github.com/zsc000722/ppt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"3d-parameter-efficient-fine-tuning-for","task_name":"3D Parameter-Efficient Fine-Tuning for Classification"},{"task_slug":"3d-point-cloud-classification","task_name":"3D Point Cloud Classification"},{"task_slug":"point-cloud-classification","task_name":"Point Cloud Classification"},{"task_slug":null,"task_name":"Position"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"parameter-efficient-fine-tuning","task_name":"parameter-efficient fine-tuning"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/3d-point-cloud-classification-on-modelnet40","task":"3D Point Cloud Classification","dataset":"ModelNet40","model":"PointMAE+PPT","rank_in_archive_order":38,"of":111,"metrics":{"Overall Accuracy":"93.88"},"uses_additional_data":false},{"leaderboard":"/sota/3d-point-cloud-classification-on-scanobjectnn","task":"3D Point Cloud Classification","dataset":"ScanObjectNN","model":"ReCon+PPT","rank_in_archive_order":23,"of":77,"metrics":{"OBJ-BG (OA)":"95.01","OBJ-ONLY (OA)":"93.28","Overall Accuracy":"89.52"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.11567","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}