{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simulation-based-inference-has-its-own","title":"Simulation-based inference has its own Dodelson-Schneider effect (but it knows that it does)","arxiv_id":"2412.02311","date":"2024-12-03","proceeding":null,"authors":["Jed Homer","Oliver Friedrich","Daniel Gruen"],"abstract":"Making inferences about physical properties of the Universe requires knowledge of the data likelihood. A Gaussian distribution is commonly assumed for the uncertainties with a covariance matrix estimated from a set of simulations. The noise in such covariance estimates causes two problems: it distorts the width of the parameter contours, and it adds scatter to the location of those contours which is not captured by the widths themselves. For non-Gaussian likelihoods, an approximation may be derived via Simulation-Based Inference (SBI). It is often implicitly assumed that parameter constraints from SBI analyses, which do not use covariance matrices, are not affected by the same problems as parameter estimation with a covariance matrix estimated from simulations. We investigate whether SBI suffers from effects similar to those of covariance estimation in Gaussian likelihoods. We use Neural Posterior and Likelihood Estimation with continuous and masked autoregressive normalizing flows for density estimation. We fit our approximate posterior models to simulations drawn from a Gaussian linear model, so that the SBI result can be compared to the true posterior. We test linear and neural network based compression, demonstrating that neither methods circumvent the issues of covariance estimation. SBI suffers an inflation of posterior variance that is equal or greater than the analytical result in covariance estimation for Gaussian likelihoods for the same number of simulations. The assumption that SBI requires a smaller number of simulations than covariance estimation for a Gaussian likelihood analysis is inaccurate. The limitations of traditional likelihood analysis with simulation-based covariance remain for SBI with a finite simulation budget. Despite these issues, we show that SBI correctly draws the true posterior contour given enough simulations.","url_abs":"https://arxiv.org/abs/2412.02311v1","url_pdf":"https://arxiv.org/pdf/2412.02311v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"simulation-based-inference-has-its-own","repo_url":"https://github.com/homerjed/sbiax","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.02311","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.02311"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/homerjed/sbiax","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5b6a728d49491002","entry":"batch_loss_fn","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/train/loss.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/train/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5b6a728d49491002"}},{"code_sha256_prefix":"8e5e561077c68249","entry":"default_weights","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/ndes/ensemble.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/ndes/ensemble.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e5e561077c68249"}},{"code_sha256_prefix":"371c9bb56b1ce9e1","entry":"get_time_embedder","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/ndes/cnf.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/ndes/cnf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"371c9bb56b1ce9e1"}},{"code_sha256_prefix":"137c752532221778","entry":"get_timestep_embedding","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/ndes/cnf.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/ndes/cnf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"137c752532221778"}},{"code_sha256_prefix":"9df76088cc0db06d","entry":"get_trial_hyperparameters","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/meta/_optuna.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/meta/_optuna.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9df76088cc0db06d"}},{"code_sha256_prefix":"acfde9eeb657b869","entry":"linearized_model","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/compression/linear.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/compression/linear.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acfde9eeb657b869"}},{"code_sha256_prefix":"2e6b27b854242c96","entry":"loss","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/compression/nn.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/compression/nn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2e6b27b854242c96"}},{"code_sha256_prefix":"4ca809a7d6395250","entry":"mle","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/compression/linear.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/compression/linear.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ca809a7d6395250"}},{"code_sha256_prefix":"d47f4a1a68572f25","entry":"pdf_mse_loss","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/train/loss.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/train/loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d47f4a1a68572f25"}},{"code_sha256_prefix":"a7e3a2d17a8e3b74","entry":"simulator","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/compression/linear.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/compression/linear.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7e3a2d17a8e3b74"}},{"code_sha256_prefix":"d781cea628a47da8","entry":"sort_sample","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/train/loader.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/train/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d781cea628a47da8"}},{"code_sha256_prefix":"6bdc6dfc8f73252e","entry":"stop_grad","repo":"homerjed/sbiax","repo_kind":"official","path":"sbiax/ndes/scaler.py","file_url":"https://github.com/homerjed/sbiax/blob/HEAD/sbiax/ndes/scaler.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6bdc6dfc8f73252e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}