{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compiler-optimization-via-llm-reasoning-for","title":"Compiler Optimization via LLM Reasoning for Efficient Model Serving","arxiv_id":"2506.01374","date":"2025-06-02","proceeding":null,"authors":["Sujun Tang","Christopher Priebe","Rohan Mahapatra","Lianhui Qin","Hadi Esmaeilzadeh"],"abstract":"While model serving has unlocked unprecedented capabilities, the high cost of serving large-scale models continues to be a significant barrier to widespread accessibility and rapid innovation. Compiler optimizations have long driven substantial performance improvements, but existing compilers struggle with neural workloads due to the exponentially large and highly interdependent space of possible transformations. Although existing stochastic search techniques can be effective, they are often sample-inefficient and fail to leverage the structural context underlying compilation decisions. We set out to investigate the research question of whether reasoning with large language models (LLMs), without any retraining, can leverage the context-aware decision space of compiler optimization to significantly improve sample efficiency. To that end, we introduce a novel compilation framework (dubbed REASONING COMPILER) that formulates optimization as a sequential, context-aware decision process, guided by a large language model and structured Monte Carlo tree search (MCTS). The LLM acts as a proposal mechanism, suggesting hardware-aware transformations that reflect the current program state and accumulated performance feedback. Monte Carlo tree search (MCTS) incorporates the LLM-generated proposals to balance exploration and exploitation, facilitating structured, context-sensitive traversal of the expansive compiler optimization space. By achieving substantial speedups with markedly fewer samples than leading neural compilers, our approach demonstrates the potential of LLM-guided reasoning to transform the landscape of compiler optimization.","url_abs":"https://arxiv.org/abs/2506.01374v1","url_pdf":"https://arxiv.org/pdf/2506.01374v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compiler-optimization-via-llm-reasoning-for","repo_url":"https://github.com/Anna-Bele/LLM_MCTS_Search","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"compiler-optimization","task_name":"Compiler Optimization"},{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.01374","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.01374"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/he-actlab/REASONING_COMPILER","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Anna-Bele/LLM_MCTS_Search","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":6},"by_repo_kind":{"named_in_paper":{"samples":1,"ran":0,"repositories":1},"found_in_text":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"92355c2360a14ceb","entry":"change_type","repo":"he-actlab/REASONING_COMPILER","repo_kind":"found_in_text","path":"ci/jenkins/generate.py","file_url":"https://github.com/he-actlab/REASONING_COMPILER/blob/HEAD/ci/jenkins/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92355c2360a14ceb"}},{"code_sha256_prefix":"416247bc8c60fa6a","entry":"lines_without_generated_tag","repo":"he-actlab/REASONING_COMPILER","repo_kind":"found_in_text","path":"ci/jenkins/generate.py","file_url":"https://github.com/he-actlab/REASONING_COMPILER/blob/HEAD/ci/jenkins/generate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"416247bc8c60fa6a"}},{"code_sha256_prefix":"833de9a5dfdc5c20","entry":"load_module","repo":"he-actlab/REASONING_COMPILER","repo_kind":"found_in_text","path":"python/tvm/runtime/module.py","file_url":"https://github.com/he-actlab/REASONING_COMPILER/blob/HEAD/python/tvm/runtime/module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"833de9a5dfdc5c20"}},{"code_sha256_prefix":"0ae10ec0399c40ac","entry":"load_static_library","repo":"he-actlab/REASONING_COMPILER","repo_kind":"found_in_text","path":"python/tvm/runtime/module.py","file_url":"https://github.com/he-actlab/REASONING_COMPILER/blob/HEAD/python/tvm/runtime/module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ae10ec0399c40ac"}},{"code_sha256_prefix":"bea3ea982f5980fc","entry":"parse_semver","repo":"Anna-Bele/LLM_MCTS_Search","repo_kind":"named_in_paper","path":"python/gen_requirements.py","file_url":"https://github.com/Anna-Bele/LLM_MCTS_Search/blob/HEAD/python/gen_requirements.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bea3ea982f5980fc"}},{"code_sha256_prefix":"8c09f14919be3f2b","entry":"system_lib","repo":"he-actlab/REASONING_COMPILER","repo_kind":"found_in_text","path":"python/tvm/runtime/module.py","file_url":"https://github.com/he-actlab/REASONING_COMPILER/blob/HEAD/python/tvm/runtime/module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c09f14919be3f2b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}