{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-language-models-are-neurosymbolic","title":"Large Language Models Are Neurosymbolic Reasoners","arxiv_id":"2401.09334","date":"2024-01-17","proceeding":null,"authors":["Meng Fang","Shilong Deng","Yudi Zhang","Zijing Shi","Ling Chen","Mykola Pechenizkiy","Jun Wang"],"abstract":"A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of Large Language Models (LLMs) as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.","url_abs":"https://arxiv.org/abs/2401.09334v1","url_pdf":"https://arxiv.org/pdf/2401.09334v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-language-models-are-neurosymbolic","repo_url":"https://github.com/hyintell/llmsymbolic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"math","task_name":"Math"},{"task_slug":"text-based-games","task_name":"text-based games"},{"task_slug":null,"task_name":"valid"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2401.09334","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.09334"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hyintell/llmsymbolic","reach":{"status":"ok"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"349de02d458cce50","entry":"llm","repo":"hyintell/llmsymbolic","repo_kind":"official","path":"play_game_by_LLM.py","file_url":"https://github.com/hyintell/llmsymbolic/blob/HEAD/play_game_by_LLM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"349de02d458cce50"}},{"code_sha256_prefix":"26e76a7133383454","entry":"llm_multiple_round","repo":"hyintell/llmsymbolic","repo_kind":"official","path":"play_game_by_LLM.py","file_url":"https://github.com/hyintell/llmsymbolic/blob/HEAD/play_game_by_LLM.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"26e76a7133383454"}},{"code_sha256_prefix":"1ccfbda7f072be06","entry":"retry","repo":"hyintell/llmsymbolic","repo_kind":"official","path":"utils/wandb_utils.py","file_url":"https://github.com/hyintell/llmsymbolic/blob/HEAD/utils/wandb_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ccfbda7f072be06"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}