{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/nonlinear-ensemble-filtering-with-diffusion","title":"Nonlinear ensemble filtering with diffusion models: Application to the surface quasi-geostrophic dynamics","arxiv_id":"2404.00844","date":"2024-04-01","proceeding":null,"authors":["Feng Bao","Hristo G. Chipilski","Siming Liang","Guannan Zhang","Jeffrey S. Whitaker"],"abstract":"The intersection between classical data assimilation methods and novel machine learning techniques has attracted significant interest in recent years. Here we explore another promising solution in which diffusion models are used to formulate a robust nonlinear ensemble filter for sequential data assimilation. Unlike standard machine learning methods, the proposed \\textit{Ensemble Score Filter (EnSF)} is completely training-free and can efficiently generate a set of analysis ensemble members. In this study, we apply the EnSF to a surface quasi-geostrophic model and compare its performance against the popular Local Ensemble Transform Kalman Filter (LETKF), which makes Gaussian assumptions on the posterior distribution. Numerical tests demonstrate that EnSF maintains stable performance in the absence of localization and for a variety of experimental settings. We find that EnSF achieves competitive performance relative to LETKF in the case of linear observations, but leads to significant advantages when the state is nonlinearly observed and the numerical model is subject to unexpected shocks. A spectral decomposition of the analysis results shows that the largest improvements over LETKF occur at large scales (small wavenumbers) where LETKF lacks sufficient ensemble spread. Overall, this initial application of EnSF to a geophysical model of intermediate complexity is very encouraging, and motivates further developments of the algorithm for more realistic problems.","url_abs":"https://arxiv.org/abs/2404.00844v1","url_pdf":"https://arxiv.org/pdf/2404.00844v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"nonlinear-ensemble-filtering-with-diffusion","repo_url":"https://github.com/jswhit/sqgturb","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.00844","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}