{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-efficient-operator-learning-via","title":"Data-Efficient Operator Learning via Unsupervised Pretraining and In-Context Learning","arxiv_id":"2402.15734","date":"2024-02-24","proceeding":null,"authors":["Wuyang Chen","Jialin Song","Pu Ren","Shashank Subramanian","Dmitriy Morozov","Michael W. Mahoney"],"abstract":"Recent years have witnessed the promise of coupling machine learning methods and physical domain-specific insights for solving scientific problems based on partial differential equations (PDEs). However, being data-intensive, these methods still require a large amount of PDE data. This reintroduces the need for expensive numerical PDE solutions, partially undermining the original goal of avoiding these expensive simulations. In this work, seeking data efficiency, we design unsupervised pretraining for PDE operator learning. To reduce the need for training data with heavy simulation costs, we mine unlabeled PDE data without simulated solutions, and we pretrain neural operators with physics-inspired reconstruction-based proxy tasks. To improve out-of-distribution performance, we further assist neural operators in flexibly leveraging a similarity-based method that learns in-context examples, without incurring extra training costs or designs. Extensive empirical evaluations on a diverse set of PDEs demonstrate that our method is highly data-efficient, more generalizable, and even outperforms conventional vision-pretrained models. We provide our code at https://github.com/delta-lab-ai/data_efficient_nopt.","url_abs":"https://arxiv.org/abs/2402.15734v3","url_pdf":"https://arxiv.org/pdf/2402.15734v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-efficient-operator-learning-via","repo_url":"https://github.com/delta-lab-ai/data_efficient_nopt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"operator-learning","task_name":"Operator learning"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.15734","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.15734"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/delta-lab-ai/data_efficient_nopt","reach":null}],"summary":{"ran_draft_wrong":3,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"f7286d5a53b111fe","entry":"add_weight_decay","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f7286d5a53b111fe"}},{"code_sha256_prefix":"7591530c94cb82e1","entry":"grad_clone","repo":"delta-lab-ai/data_efficient_nopt","repo_kind":"official","path":"pretrain_basic.py","file_url":"https://github.com/delta-lab-ai/data_efficient_nopt/blob/HEAD/pretrain_basic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7591530c94cb82e1"}},{"code_sha256_prefix":"630e6bdb88095b73","entry":"grad_norm","repo":"delta-lab-ai/data_efficient_nopt","repo_kind":"official","path":"train_basic.py","file_url":"https://github.com/delta-lab-ai/data_efficient_nopt/blob/HEAD/train_basic.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"630e6bdb88095b73"}},{"code_sha256_prefix":"3cc1d78804edbcdc","entry":"l2_err","repo":"delta-lab-ai/data_efficient_nopt","repo_kind":"official","path":"train_basic.py","file_url":"https://github.com/delta-lab-ai/data_efficient_nopt/blob/HEAD/train_basic.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cc1d78804edbcdc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}