{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2606-12691","title":"Two-Layer Linear Auto-Regressive Models Estimate Latent States","arxiv_id":"2606.12691","date":"2026-06-10","proceeding":"ICML","authors":["Yahya Sattar","Sunmook Choi","Leo Maynard-Zhang","Yassir Jedra","Maryam Fazel","Sarah Dean"],"abstract":"Auto-regressive models have emerged as powerful tools for sequential data, from language to video. Understanding how and why these models learn latent representations remains an open theoretical question. In this work, we demonstrate that when trained by empirical risk minimization on data from partially observed linear dynamical systems, two-layer linear auto-regressive models naturally learn to approximate Kalman filtering. In particular, we show that the learned hidden representation coincides, up to a similarity transformation, with the state estimates produced by the optimal (Kalman) filter, even though the model has no explicit knowledge of the underlying dynamics or state. The result follows from three main insights. First, we establish that the Kalman filter is well approximated by an auto-regressive model with bounded truncation error. Second, we show that despite non-convexity, the two-layer optimization landscape is benign, i.e., all stationary points are either strict saddles or global minima. Finally, as our main contributions, we provide finite-sample guarantees on prediction error, parameter estimation error, and latent state recovery. Numerical simulations support the theoretical results and demonstrate that the latent representations of auto-regressive models recover state estimates.","url_abs":"https://arxiv.org/abs/2606.12691","url_pdf":"https://arxiv.org/pdf/2606.12691","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2606.12691","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2606.12691"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/sdean-group/linear-ar-kf","reach":null}],"summary":{"ran":8,"unverified":4},"by_repo_kind":{"found_in_text":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"30ade6ec01fc4a9a","entry":"build_autoregressive_arrays","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/dataloader.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30ade6ec01fc4a9a"}},{"code_sha256_prefix":"e3396e7b20281c30","entry":"is_observable","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/define_system.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/define_system.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3396e7b20281c30"}},{"code_sha256_prefix":"0607ad3f476bbd0a","entry":"make_dataset","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/dataloader.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0607ad3f476bbd0a"}},{"code_sha256_prefix":"d0b6d3e72e8f3dce","entry":"observability_matrix","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/define_system.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/define_system.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d0b6d3e72e8f3dce"}},{"code_sha256_prefix":"2cb9b36fdaf68552","entry":"run_steady_state_kalman_filter","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/steady_state_KF.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/steady_state_KF.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2cb9b36fdaf68552"}},{"code_sha256_prefix":"1ea117368c6164e1","entry":"simulate_controlgym_linear_system","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/controlgym_adapter.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/controlgym_adapter.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ea117368c6164e1"}},{"code_sha256_prefix":"8558bb12edab260f","entry":"spectral_radius","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/define_system.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/define_system.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8558bb12edab260f"}},{"code_sha256_prefix":"ad038e11ba63c33f","entry":"steady_state_kalman_gain","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/steady_state_KF.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/steady_state_KF.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad038e11ba63c33f"}},{"code_sha256_prefix":"b24401263e24ac80","entry":"load_controlgym_linear_system","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/controlgym_adapter.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/controlgym_adapter.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b24401263e24ac80"}},{"code_sha256_prefix":"1ac03026acd76a9f","entry":"parameter_search","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/architecture_search.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/architecture_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ac03026acd76a9f"}},{"code_sha256_prefix":"40f64b88d20cef86","entry":"screen_controlgym_linear_systems","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/controlgym_adapter.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/controlgym_adapter.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40f64b88d20cef86"}},{"code_sha256_prefix":"7bca755be1fb12fd","entry":"simulate","repo":"sdean-group/linear-ar-kf","repo_kind":"found_in_text","path":"exp/simulate.py","file_url":"https://github.com/sdean-group/linear-ar-kf/blob/HEAD/exp/simulate.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7bca755be1fb12fd"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}