{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/provable-ordering-and-continuity-in-vision","title":"Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents","arxiv_id":"2502.01218","date":"2025-02-03","proceeding":null,"authors":["Zhizhen Zhang","Lei Zhu","Zhen Fang","Zi Huang","Yadan Luo"],"abstract":"Pre-training vision-language representations on human action videos has emerged as a promising approach to reduce reliance on large-scale expert demonstrations for training embodied agents. However, prior methods often employ time contrastive learning based on goal-reaching heuristics, progressively aligning language instructions from the initial to the final frame. This overemphasis on future frames can result in erroneous vision-language associations, as actions may terminate early or include irrelevant moments in the end. To address this issue, we propose Action Temporal Coherence Learning (AcTOL) to learn ordered and continuous vision-language representations without rigid goal-based constraint. AcTOL treats a video as a continuous trajectory where it (1) contrasts semantic differences between frames to reflect their natural ordering, and (2) imposes a local Brownian bridge constraint to ensure smooth transitions across intermediate frames. Extensive imitation learning experiments on both simulated and real robots show that the pretrained features significantly enhance downstream manipulation tasks with high robustness to different linguistic styles of instructions, offering a viable pathway toward generalized embodied agents.","url_abs":"https://arxiv.org/abs/2502.01218v2","url_pdf":"https://arxiv.org/pdf/2502.01218v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"provable-ordering-and-continuity-in-vision","repo_url":"https://github.com/daisy-zzz/actol","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"imitation-learning","task_name":"Imitation Learning"}],"methods":[{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.01218","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.01218"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/daisy-zzz/actol","reach":null}],"summary":{"ran":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"a7a6afa631c4d854","entry":"LabelDifference","repo":"daisy-zzz/actol","repo_kind":"official","path":"AcTOL/losses.py","file_url":"https://github.com/daisy-zzz/actol/blob/HEAD/AcTOL/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a7a6afa631c4d854"}},{"code_sha256_prefix":"e2e676172521103e","entry":"brownian_bridge_loss","repo":"daisy-zzz/actol","repo_kind":"official","path":"AcTOL/losses.py","file_url":"https://github.com/daisy-zzz/actol/blob/HEAD/AcTOL/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e2e676172521103e"}},{"code_sha256_prefix":"375a840c64f70adc","entry":"AcTOL_loss","repo":"daisy-zzz/actol","repo_kind":"official","path":"AcTOL/losses.py","file_url":"https://github.com/daisy-zzz/actol/blob/HEAD/AcTOL/losses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"375a840c64f70adc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}