{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/masked-trajectory-models-for-prediction","title":"Masked Trajectory Models for Prediction, Representation, and Control","arxiv_id":"2305.02968","date":"2023-05-04","proceeding":null,"authors":["Philipp Wu","Arjun Majumdar","Kevin Stone","Yixin Lin","Igor Mordatch","Pieter Abbeel","Aravind Rajeswaran"],"abstract":"We introduce Masked Trajectory Models (MTM) as a generic abstraction for sequential decision making. MTM takes a trajectory, such as a state-action sequence, and aims to reconstruct the trajectory conditioned on random subsets of the same trajectory. By training with a highly randomized masking pattern, MTM learns versatile networks that can take on different roles or capabilities, by simply choosing appropriate masks at inference time. For example, the same MTM network can be used as a forward dynamics model, inverse dynamics model, or even an offline RL agent. Through extensive experiments in several continuous control tasks, we show that the same MTM network -- i.e. same weights -- can match or outperform specialized networks trained for the aforementioned capabilities. Additionally, we find that state representations learned by MTM can significantly accelerate the learning speed of traditional RL algorithms. Finally, in offline RL benchmarks, we find that MTM is competitive with specialized offline RL algorithms, despite MTM being a generic self-supervised learning method without any explicit RL components. Code is available at https://github.com/facebookresearch/mtm","url_abs":"https://arxiv.org/abs/2305.02968v1","url_pdf":"https://arxiv.org/pdf/2305.02968v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"masked-trajectory-models-for-prediction","repo_url":"https://github.com/facebookresearch/mtm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"decision-making","task_name":"Decision Making"},{"task_slug":"offline-rl","task_name":"Offline RL"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"sequential-decision-making","task_name":"Sequential Decision Making"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.02968","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.02968"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/mtm","reach":null}],"summary":{"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"efa8e5ed19b69df0","entry":"get_1d_sincos_pos_embed_from_grid","repo":"facebookresearch/mtm","repo_kind":"official","path":"research/mtm/models/mtm_model.py","file_url":"https://github.com/facebookresearch/mtm/blob/HEAD/research/mtm/models/mtm_model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"efa8e5ed19b69df0"}},{"code_sha256_prefix":"1d8180c8452d04c6","entry":"MTM","repo":"facebookresearch/mtm","repo_kind":"official","path":"research/mtm/models/mtm_model.py","file_url":"https://github.com/facebookresearch/mtm/blob/HEAD/research/mtm/models/mtm_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d8180c8452d04c6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}