{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2602-21307","title":"SymTorch: Symbolic Distillation of Neural Networks","arxiv_id":"2602.21307","date":"2026-02-24","proceeding":null,"authors":["Elizabeth S. Z. Tan","Adil Soubki","Miles Cranmer"],"abstract":"What mathematical functions do neural network components learn? Symbolic distillation addresses this question by expressing neural network components with interpretable, closed-form mathematical expressions that expose the functional structure learned during training. We develop symbolic distillation as a systematic, architecture-agnostic methodology, and release our approach as the open-source SymTorch package - a PySR-powered library built natively for the PyTorch ecosystem. Applying this methodology across diverse architectures, we find that SymTorch is successful in the automated discovery of physical laws. Specifically, our approach (1) recovers pairwise interaction forces from graph neural networks trained on empirical $n$-body observations, (2) distills the exact closed-form PDE/ODE solutions of multiple physical systems, including the value of constants, from physics-informed neural networks trained on sparse data, and (3) uncovers the chaotic dynamics of the Lorenz system from high-dimensional data, ultimately outperforming the base neural network on downstream prediction tasks. We further demonstrate the utility of our framework for model interpretability by providing an optimized implementation of SLIME - a symbolic extension to the LIME explainability method. SLIME consistently outperforms LIME across predictive metrics across eight popular classification and regression benchmarks, while still providing an interpretable local symbolic model. Lastly, we investigate replacing transformer MLP layers with symbolic surrogates: replacing 1-7 layers with symbolic approximations yields 2-19\\% throughput improvements and up to 18.7\\% VRAM reduction, with the resulting hybrid models lying on the Pareto front of throughput versus perplexity among open-source LLMs of comparable scale.","url_abs":"https://arxiv.org/abs/2602.21307","url_pdf":"https://arxiv.org/pdf/2602.21307","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2602.21307","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2602.21307"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/astroautomata/SymTorch","reach":null}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":7,"samples":[{"code_sha256_prefix":"55fe59f8c1b45669","entry":"benchmark_model","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"benchmarking/benchmark_symbolicmodel.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/benchmarking/benchmark_symbolicmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55fe59f8c1b45669"}},{"code_sha256_prefix":"7bf060f51f82862c","entry":"f_func","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"case_studies/pruning_demo/pruning_script.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/case_studies/pruning_demo/pruning_script.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7bf060f51f82862c"}},{"code_sha256_prefix":"5cd04b1574dc0d9d","entry":"g_func","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"case_studies/pruning_demo/pruning_script.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/case_studies/pruning_demo/pruning_script.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5cd04b1574dc0d9d"}},{"code_sha256_prefix":"2514f8d216dd38d2","entry":"make_2d_hists","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"case_studies/pinn_demo/pinn_script.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/case_studies/pinn_demo/pinn_script.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2514f8d216dd38d2"}},{"code_sha256_prefix":"4464c5b0ea06af01","entry":"temp","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"case_studies/pinn_demo/pinn_script.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/case_studies/pinn_demo/pinn_script.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4464c5b0ea06af01"}},{"code_sha256_prefix":"c470692c665eb0c0","entry":"train","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"case_studies/pinn_demo/pinn_script.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/case_studies/pinn_demo/pinn_script.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c470692c665eb0c0"}},{"code_sha256_prefix":"b6d4a688506f0578","entry":"train_model","repo":"astroautomata/SymTorch","repo_kind":"found_in_text","path":"benchmarking/benchmark_symbolicmodel.py","file_url":"https://github.com/astroautomata/SymTorch/blob/HEAD/benchmarking/benchmark_symbolicmodel.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b6d4a688506f0578"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}