{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uni3d-exploring-unified-3d-representation-at","title":"Uni3D: Exploring Unified 3D Representation at Scale","arxiv_id":"2310.06773","date":"2023-10-10","proceeding":null,"authors":["Junsheng Zhou","Jinsheng Wang","Baorui Ma","Yu-Shen Liu","Tiejun Huang","Xinlong Wang"],"abstract":"Scaling up representations for images or text has been extensively investigated in the past few years and has led to revolutions in learning vision and language. However, scalable representation for 3D objects and scenes is relatively unexplored. In this work, we present Uni3D, a 3D foundation model to explore the unified 3D representation at scale. Uni3D uses a 2D initialized ViT end-to-end pretrained to align the 3D point cloud features with the image-text aligned features. Via the simple architecture and pretext task, Uni3D can leverage abundant 2D pretrained models as initialization and image-text aligned models as the target, unlocking the great potential of 2D models and scaling-up strategies to the 3D world. We efficiently scale up Uni3D to one billion parameters, and set new records on a broad range of 3D tasks, such as zero-shot classification, few-shot classification, open-world understanding and part segmentation. We show that the strong Uni3D representation also enables applications such as 3D painting and retrieval in the wild. We believe that Uni3D provides a new direction for exploring both scaling up and efficiency of the representation in 3D domain.","url_abs":"https://arxiv.org/abs/2310.06773v1","url_pdf":"https://arxiv.org/pdf/2310.06773v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uni3d-exploring-unified-3d-representation-at","repo_url":"https://github.com/baaivision/uni3d","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"uni3d-exploring-unified-3d-representation-at","repo_url":"https://github.com/baaivision/GeoDream","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null}],"tasks":[{"task_slug":"3d-object-classification","task_name":"3D Object Classification"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"},{"task_slug":"zero-shot-transfer-3d-point-cloud","task_name":"Zero-Shot Transfer 3D Point Cloud Classification"},{"task_slug":"zero-shot-3d-point-cloud-classification","task_name":"Zero-shot 3D Point Cloud Classification"},{"task_slug":"zero-shot-3d-classification","task_name":"Zero-shot 3D classification"},{"task_slug":"zero-shot-classification-unified-classes","task_name":"Zero-shot Classification (unified classes)"},{"task_slug":null,"task_name":"zero-shot-classification"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/zero-shot-transfer-3d-point-cloud","task":"Zero-Shot Transfer 3D Point Cloud Classification","dataset":"ModelNet40","model":"Uni3D","rank_in_archive_order":1,"of":16,"metrics":{"Accuracy (%)":"88.2"},"uses_additional_data":true},{"leaderboard":"/sota/zero-shot-transfer-3d-point-cloud-2","task":"Zero-Shot Transfer 3D Point Cloud Classification","dataset":"ScanObjectNN","model":"Uni3D","rank_in_archive_order":2,"of":10,"metrics":{"OBJ_ONLY Accuracy(%)":"65.3"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2310.06773","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.06773"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baaivision/GeoDream","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baaivision/uni3d","reach":null}],"summary":{"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c0c203033ea34986","entry":"get_metric_names","repo":"baaivision/uni3d","repo_kind":"official","path":"models/uni3d.py","file_url":"https://github.com/baaivision/uni3d/blob/HEAD/models/uni3d.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0c203033ea34986"}},{"code_sha256_prefix":"426b192fc7d8eac4","entry":"get_filter_loss","repo":"baaivision/uni3d","repo_kind":"official","path":"models/uni3d.py","file_url":"https://github.com/baaivision/uni3d/blob/HEAD/models/uni3d.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"426b192fc7d8eac4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}