{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-efficient-learning-with-neural-programs","title":"Data-Efficient Learning with Neural Programs","arxiv_id":"2406.06246","date":"2024-06-10","proceeding":null,"authors":["Alaia Solko-Breslin","Seewon Choi","Ziyang Li","Neelay Velingker","Rajeev Alur","Mayur Naik","Eric Wong"],"abstract":"Many computational tasks can be naturally expressed as a composition of a DNN followed by a program written in a traditional programming language or an API call to an LLM. We call such composites \"neural programs\" and focus on the problem of learning the DNN parameters when the training data consist of end-to-end input-output labels for the composite. When the program is written in a differentiable logic programming language, techniques from neurosymbolic learning are applicable, but in general, the learning for neural programs requires estimating the gradients of black-box components. We present an algorithm for learning neural programs, called ISED, that only relies on input-output samples of black-box components. For evaluation, we introduce new benchmarks that involve calls to modern LLMs such as GPT-4 and also consider benchmarks from the neurosymbolic learning literature. Our evaluation shows that for the latter benchmarks, ISED has comparable performance to state-of-the-art neurosymbolic frameworks. For the former, we use adaptations of prior work on gradient approximations of black-box components as a baseline, and show that ISED achieves comparable accuracy but in a more data- and sample-efficient manner.","url_abs":"https://arxiv.org/abs/2406.06246v2","url_pdf":"https://arxiv.org/pdf/2406.06246v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-efficient-learning-with-neural-programs","repo_url":"https://github.com/alaiasolkobreslin/ised","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.06246","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.06246"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alaiasolkobreslin/ised","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"79d53604e4138cba","entry":"id","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"generation-pipeline/structured_dataset.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/generation-pipeline/structured_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"79d53604e4138cba"}},{"code_sha256_prefix":"dec59c6ad7903300","entry":"sum_2","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"generation-pipeline/task_program.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/generation-pipeline/task_program.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dec59c6ad7903300"}},{"code_sha256_prefix":"2961394ddf070950","entry":"sum_3","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"generation-pipeline/task_program.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/generation-pipeline/task_program.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2961394ddf070950"}},{"code_sha256_prefix":"a22d822533f1f418","entry":"sum_4","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"generation-pipeline/task_program.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/generation-pipeline/task_program.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a22d822533f1f418"}},{"code_sha256_prefix":"39b961f06732092a","entry":"compute_loss","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"custom/sudoku/src/ised.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/custom/sudoku/src/ised.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39b961f06732092a"}},{"code_sha256_prefix":"078a4055138f52d3","entry":"vectorize","repo":"alaiasolkobreslin/ised","repo_kind":"official","path":"custom/sudoku/src/ised.py","file_url":"https://github.com/alaiasolkobreslin/ised/blob/HEAD/custom/sudoku/src/ised.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"078a4055138f52d3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}