{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/latent-action-learning-requires-supervision","title":"Latent Action Learning Requires Supervision in the Presence of Distractors","arxiv_id":"2502.00379","date":"2025-02-01","proceeding":null,"authors":["Alexander Nikulin","Ilya Zisman","Denis Tarasov","Nikita Lyubaykin","Andrei Polubarov","Igor Kiselev","Vladislav Kurenkov"],"abstract":"Recently, latent action learning, pioneered by Latent Action Policies (LAPO), have shown remarkable pre-training efficiency on observation-only data, offering potential for leveraging vast amounts of video available on the web for embodied AI. However, prior work has focused on distractor-free data, where changes between observations are primarily explained by ground-truth actions. Unfortunately, real-world videos contain action-correlated distractors that may hinder latent action learning. Using Distracting Control Suite (DCS) we empirically investigate the effect of distractors on latent action learning and demonstrate that LAPO struggle in such scenario. We propose LAOM, a simple LAPO modification that improves the quality of latent actions by 8x, as measured by linear probing. Importantly, we show that providing supervision with ground-truth actions, as few as 2.5% of the full dataset, during latent action learning improves downstream performance by 4.2x on average. Our findings suggest that integrating supervision during Latent Action Models (LAM) training is critical in the presence of distractors, challenging the conventional pipeline of first learning LAM and only then decoding from latent to ground-truth actions.","url_abs":"https://arxiv.org/abs/2502.00379v4","url_pdf":"https://arxiv.org/pdf/2502.00379v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.00379","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.00379"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/dunnolab/laom","reach":null}],"summary":{"ran":6,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":7,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"895458a98fc89d34","entry":"EncoderBlock","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"895458a98fc89d34"}},{"code_sha256_prefix":"0f86addf32976907","entry":"LAOM","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0f86addf32976907"}},{"code_sha256_prefix":"69926a59ac9559fa","entry":"LatentActHead","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"69926a59ac9559fa"}},{"code_sha256_prefix":"cf521774c77dd137","entry":"LatentObsHead","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf521774c77dd137"}},{"code_sha256_prefix":"4efde63d825cc235","entry":"MLPBlock","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4efde63d825cc235"}},{"code_sha256_prefix":"a7066bdc1c837aa5","entry":"ResidualBlock","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7066bdc1c837aa5"}},{"code_sha256_prefix":"92b3b815b957a079","entry":"weight_init","repo":"dunnolab/laom","repo_kind":"found_in_text","path":"src/nn.py","file_url":"https://github.com/dunnolab/laom/blob/HEAD/src/nn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92b3b815b957a079"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}