{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/space-time-correspondence-as-a-contrastive","title":"Space-Time Correspondence as a Contrastive Random Walk","arxiv_id":"2006.14613","date":"2020-06-25","proceeding":"NeurIPS 2020 12","authors":["Allan Jabri","Andrew Owens","Alexei A. Efros"],"abstract":"This paper proposes a simple self-supervised approach for learning a representation for visual correspondence from raw video. We cast correspondence as prediction of links in a space-time graph constructed from video. In this graph, the nodes are patches sampled from each frame, and nodes adjacent in time can share a directed edge. We learn a representation in which pairwise similarity defines transition probability of a random walk, so that long-range correspondence is computed as a walk along the graph. We optimize the representation to place high probability along paths of similarity. Targets for learning are formed without supervision, by cycle-consistency: the objective is to maximize the likelihood of returning to the initial node when walking along a graph constructed from a palindrome of frames. Thus, a single path-level constraint implicitly supervises chains of intermediate comparisons. When used as a similarity metric without adaptation, the learned representation outperforms the self-supervised state-of-the-art on label propagation tasks involving objects, semantic parts, and pose. Moreover, we demonstrate that a technique we call edge dropout, as well as self-supervised adaptation at test-time, further improve transfer for object-centric correspondence.","url_abs":"https://arxiv.org/abs/2006.14613v2","url_pdf":"https://arxiv.org/pdf/2006.14613v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"space-time-correspondence-as-a-contrastive","repo_url":"https://github.com/ajabri/videowalk","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"dense-pixel-correspondence-estimation","task_name":"Dense Pixel Correspondence Estimation"},{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2006.14613","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2006.14613"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ajabri/videowalk","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran":3,"unverified":10},"by_repo_kind":{"official":{"samples":14,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d86fb168246b16a4","entry":"fast_hist","repo":"ajabri/videowalk","repo_kind":"official","path":"code/eval/eval_vip.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/eval/eval_vip.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d86fb168246b16a4"}},{"code_sha256_prefix":"fb2519bb639a667c","entry":"get_frame_aug","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/augs.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/augs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fb2519bb639a667c"}},{"code_sha256_prefix":"14b1cc74dcb694bf","entry":"n_patches","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/augs.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/augs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14b1cc74dcb694bf"}},{"code_sha256_prefix":"600497c0fb4eb8c7","entry":"patch_grid","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/augs.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/augs.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"600497c0fb4eb8c7"}},{"code_sha256_prefix":"cd7e0dea3e1a7fd0","entry":"common_args","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/arguments.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/arguments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd7e0dea3e1a7fd0"}},{"code_sha256_prefix":"11ed14b5cc00d18c","entry":"computePCK","repo":"ajabri/videowalk","repo_kind":"official","path":"code/eval/eval_pck.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/eval/eval_pck.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"11ed14b5cc00d18c"}},{"code_sha256_prefix":"3acc87c8ff9403a5","entry":"compute_hist","repo":"ajabri/videowalk","repo_kind":"official","path":"code/eval/eval_vip.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/eval/eval_vip.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3acc87c8ff9403a5"}},{"code_sha256_prefix":"fe5e025dcc4db6f3","entry":"draw_matches","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/visualize.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/visualize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe5e025dcc4db6f3"}},{"code_sha256_prefix":"57e8ff198fc0ac02","entry":"get_frame_transform","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/kornia_augs.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/kornia_augs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"57e8ff198fc0ac02"}},{"code_sha256_prefix":"68336aa7c98a4d33","entry":"make_gif","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/visualize.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/visualize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68336aa7c98a4d33"}},{"code_sha256_prefix":"b4eaa298ffce395e","entry":"patch_grid","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/kornia_augs.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/kornia_augs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4eaa298ffce395e"}},{"code_sha256_prefix":"1b4515fda7085b63","entry":"pca_feats","repo":"ajabri/videowalk","repo_kind":"official","path":"code/utils/visualize.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/utils/visualize.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b4515fda7085b63"}},{"code_sha256_prefix":"4a4432bf960d0dbe","entry":"resnet18","repo":"ajabri/videowalk","repo_kind":"official","path":"code/resnet.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4a4432bf960d0dbe"}},{"code_sha256_prefix":"94b05476d48954e9","entry":"resnet50","repo":"ajabri/videowalk","repo_kind":"official","path":"code/resnet.py","file_url":"https://github.com/ajabri/videowalk/blob/HEAD/code/resnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94b05476d48954e9"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}