{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tensor-var-variational-data-assimilation-in","title":"Tensor-Var: Variational Data Assimilation in Tensor Product Feature Space","arxiv_id":"2501.13312","date":"2025-01-23","proceeding":null,"authors":["Yiming Yang","Xiaoyuan Cheng","Daniel Giles","Sibo Cheng","Yi He","Xiao Xue","Boli Chen","Yukun Hu"],"abstract":"Variational data assimilation estimates the dynamical system states by minimizing a cost function that fits the numerical models with observational data. The widely used method, four-dimensional variational assimilation (4D-Var), has two primary challenges: (1) computationally demanding for complex nonlinear systems and (2) relying on state-observation mappings, which are often not perfectly known. Deep learning (DL) has been used as a more expressive class of efficient model approximators to address these challenges. However, integrating such models into 4D-Var remains challenging due to their inherent nonlinearities and the lack of theoretical guarantees for consistency in assimilation results. In this paper, we propose \\textit{Tensor-Var} to address these challenges using kernel Conditional Mean Embedding (CME). Tensor-Var improves optimization efficiency by characterizing system dynamics and state-observation mappings as linear operators, leading to a convex cost function in the feature space. Furthermore, our method provides a new perspective to incorporate CME into 4D-Var, offering theoretical guarantees of consistent assimilation results between the original and feature spaces. To improve scalability, we propose a method to learn deep features (DFs) using neural networks within the Tensor-Var framework. Experiments on chaotic systems and global weather prediction with real-time observations show that Tensor-Var outperforms conventional and DL hybrid 4D-Var baselines in accuracy while achieving efficiency comparable to the static 3D-Var method.","url_abs":"https://arxiv.org/abs/2501.13312v1","url_pdf":"https://arxiv.org/pdf/2501.13312v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tensor-var-variational-data-assimilation-in","repo_url":"https://github.com/yyimingucl/TensorVar","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.13312","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.13312"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yyimingucl/TensorVar","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":2,"unverified":2},"by_repo_kind":{"official":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bd1f17e427bf51a5","entry":"dict2namespace","repo":"yyimingucl/TensorVar","repo_kind":"official","path":"utils.py","file_url":"https://github.com/yyimingucl/TensorVar/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd1f17e427bf51a5"}},{"code_sha256_prefix":"82a15cc1e46f7e4d","entry":"to_3d","repo":"yyimingucl/TensorVar","repo_kind":"official","path":"model/ERA5_model/transformer.py","file_url":"https://github.com/yyimingucl/TensorVar/blob/HEAD/model/ERA5_model/transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"82a15cc1e46f7e4d"}},{"code_sha256_prefix":"b20f2a5df739a59e","entry":"to_4d","repo":"yyimingucl/TensorVar","repo_kind":"official","path":"model/ERA5_model/transformer.py","file_url":"https://github.com/yyimingucl/TensorVar/blob/HEAD/model/ERA5_model/transformer.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b20f2a5df739a59e"}},{"code_sha256_prefix":"c7716e71bcb9c027","entry":"count_parameters","repo":"yyimingucl/TensorVar","repo_kind":"official","path":"utils.py","file_url":"https://github.com/yyimingucl/TensorVar/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7716e71bcb9c027"}},{"code_sha256_prefix":"58ec072caeebe023","entry":"is_symmetric","repo":"yyimingucl/TensorVar","repo_kind":"official","path":"model/ERA5_model/utils.py","file_url":"https://github.com/yyimingucl/TensorVar/blob/HEAD/model/ERA5_model/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58ec072caeebe023"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}