{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/croco-self-supervised-pre-training-for-3d","title":"CroCo: Self-Supervised Pre-training for 3D Vision Tasks by Cross-View Completion","arxiv_id":"2210.10716","date":"2022-10-19","proceeding":null,"authors":["Philippe Weinzaepfel","Vincent Leroy","Thomas Lucas","Romain Brégier","Yohann Cabon","Vaibhav Arora","Leonid Antsfeld","Boris Chidlovskii","Gabriela Csurka","Jérôme Revaud"],"abstract":"Masked Image Modeling (MIM) has recently been established as a potent pre-training paradigm. A pretext task is constructed by masking patches in an input image, and this masked content is then predicted by a neural network using visible patches as sole input. This pre-training leads to state-of-the-art performance when finetuned for high-level semantic tasks, e.g. image classification and object detection. In this paper we instead seek to learn representations that transfer well to a wide variety of 3D vision and lower-level geometric downstream tasks, such as depth prediction or optical flow estimation. Inspired by MIM, we propose an unsupervised representation learning task trained from pairs of images showing the same scene from different viewpoints. More precisely, we propose the pretext task of cross-view completion where the first input image is partially masked, and this masked content has to be reconstructed from the visible content and the second image. In single-view MIM, the masked content often cannot be inferred precisely from the visible portion only, so the model learns to act as a prior influenced by high-level semantics. In contrast, this ambiguity can be resolved with cross-view completion from the second unmasked image, on the condition that the model is able to understand the spatial relationship between the two images. Our experiments show that our pretext task leads to significantly improved performance for monocular 3D vision downstream tasks such as depth estimation. In addition, our model can be directly applied to binocular downstream tasks like optical flow or relative camera pose estimation, for which we obtain competitive results without bells and whistles, i.e., using a generic architecture without any task-specific design.","url_abs":"https://arxiv.org/abs/2210.10716v2","url_pdf":"https://arxiv.org/pdf/2210.10716v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"croco-self-supervised-pre-training-for-3d","repo_url":"https://github.com/naver/croco","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"camera-pose-estimation","task_name":"Camera Pose Estimation"},{"task_slug":"depth-estimation","task_name":"Depth Estimation"},{"task_slug":"depth-prediction","task_name":"Depth Prediction"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"optical-flow-estimation","task_name":"Optical Flow Estimation"},{"task_slug":"pose-estimation","task_name":"Pose Estimation"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"mim","method_name":"MIM"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2210.10716","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2210.10716"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/naver/croco","reach":null}],"summary":{"ran":8,"ran_draft_wrong":1,"ran_fixture":3,"unverified":3},"by_repo_kind":{"official":{"samples":15,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"e43d66926c52d73f","entry":"Attention","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"e43d66926c52d73f"}},{"code_sha256_prefix":"863ed6dfef2b047a","entry":"CrossAttention","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"863ed6dfef2b047a"}},{"code_sha256_prefix":"1615adbdcbbbfe4f","entry":"DecoderBlock","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1615adbdcbbbfe4f"}},{"code_sha256_prefix":"99b1932affbf5fdf","entry":"DropPath","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"99b1932affbf5fdf"}},{"code_sha256_prefix":"001c4e1dde23a54e","entry":"Mlp","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"001c4e1dde23a54e"}},{"code_sha256_prefix":"20f0e8a44c49e22d","entry":"PatchEmbed","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"20f0e8a44c49e22d"}},{"code_sha256_prefix":"3325c483d48e12ee","entry":"PositionGetter","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"3325c483d48e12ee"}},{"code_sha256_prefix":"fe5195c3233da136","entry":"RandomMask","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fe5195c3233da136"}},{"code_sha256_prefix":"cd4d36b9126e53f2","entry":"_ntuple","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cd4d36b9126e53f2"}},{"code_sha256_prefix":"f9a1900525331e0f","entry":"drop_path","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f9a1900525331e0f"}},{"code_sha256_prefix":"4bdd36eab04c3f62","entry":"get_1d_sincos_pos_embed_from_grid","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4bdd36eab04c3f62"}},{"code_sha256_prefix":"d9a2bb7a053e82c0","entry":"get_2d_sincos_pos_embed","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d9a2bb7a053e82c0"}},{"code_sha256_prefix":"d98a51333a14237d","entry":"Block","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d98a51333a14237d"}},{"code_sha256_prefix":"f473b33a00269412","entry":"CroCoNet","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"f473b33a00269412"}},{"code_sha256_prefix":"4fd80de79832745d","entry":"get_2d_sincos_pos_embed_from_grid","repo":"naver/croco","repo_kind":"official","path":"models/croco.py","file_url":"https://github.com/naver/croco/blob/HEAD/models/croco.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4fd80de79832745d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}