{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hvdistill-transferring-knowledge-from-images","title":"HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation","arxiv_id":"2403.11817","date":"2024-03-18","proceeding":null,"authors":["Sha Zhang","Jiajun Deng","Lei Bai","Houqiang Li","Wanli Ouyang","Yanyong Zhang"],"abstract":"We present a hybrid-view-based knowledge distillation framework, termed HVDistill, to guide the feature learning of a point cloud neural network with a pre-trained image network in an unsupervised man- ner. By exploiting the geometric relationship between RGB cameras and LiDAR sensors, the correspondence between the two modalities based on both image- plane view and bird-eye view can be established, which facilitates representation learning. Specifically, the image-plane correspondences can be simply ob- tained by projecting the point clouds, while the bird- eye-view correspondences can be achieved by lifting pixels to the 3D space with the predicted depths un- der the supervision of projected point clouds. The image teacher networks provide rich semantics from the image-plane view and meanwhile acquire geometric information from the bird-eye view. Indeed, image features from the two views naturally comple- ment each other and together can ameliorate the learned feature representation of the point cloud stu- dent networks. Moreover, with a self-supervised pre- trained 2D network, HVDistill requires neither 2D nor 3D annotations. We pre-train our model on nuScenes dataset and transfer it to several downstream tasks on nuScenes, SemanticKITTI, and KITTI datasets for evaluation. Extensive experimental results show that our method achieves consistent improvements over the baseline trained from scratch and significantly out- performs the existing schemes. Codes are available at git@github.com:zhangsha1024/HVDistill.git.","url_abs":"https://arxiv.org/abs/2403.11817v1","url_pdf":"https://arxiv.org/pdf/2403.11817v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hvdistill-transferring-knowledge-from-images","repo_url":"https://github.com/zhangsha1024/HVDistill","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"cg","task_name":"NER"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.11817","atlas_url":"https://app.syntology.ai/?focus=2403.11817","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.11817"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhangsha1024/HVDistill","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_violates":1,"unverified":1},"by_repo_kind":{"official":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"129a4b3603798d9e","entry":"forgiving_state_restore","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"pretrain/model_builder.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/pretrain/model_builder.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"129a4b3603798d9e"}},{"code_sha256_prefix":"044c907ab4e72406","entry":"gen_dx_bx","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"model/vtransforms_hvd.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/model/vtransforms_hvd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"044c907ab4e72406"}},{"code_sha256_prefix":"ecdf713b7926be59","entry":"iou","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"downstream/criterion.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/downstream/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ecdf713b7926be59"}},{"code_sha256_prefix":"3da12066754004fa","entry":"iou_binary","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"downstream/criterion.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/downstream/criterion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3da12066754004fa"}},{"code_sha256_prefix":"2b229c3b0d135e55","entry":"load_state_with_same_shape","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"downstream/model_builder.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/downstream/model_builder.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b229c3b0d135e55"}},{"code_sha256_prefix":"e85b5d21dc5b8382","entry":"lovasz_grad","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"downstream/criterion.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/downstream/criterion.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e85b5d21dc5b8382"}},{"code_sha256_prefix":"6788c0c08ce724da","entry":"adapt_weights","repo":"zhangsha1024/HVDistill","repo_kind":"official","path":"model/image_model_hvdistill.py","file_url":"https://github.com/zhangsha1024/HVDistill/blob/HEAD/model/image_model_hvdistill.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6788c0c08ce724da"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}