{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/soap-enhancing-efficiency-of-generated-code","title":"EffiLearner: Enhancing Efficiency of Generated Code via Self-Optimization","arxiv_id":"2405.15189","date":"2024-05-24","proceeding":null,"authors":["Dong Huang","Jianbo Dai","Han Weng","Puzhen Wu","Yuhao QING","Heming Cui","Zhijiang Guo","Jie M. Zhang"],"abstract":"Large language models (LLMs) have shown remarkable progress in code generation, but their generated code often suffers from inefficiency, resulting in longer execution times and higher memory consumption. To address this issue, we propose \\textbf{EffiLearner}, a self-optimization framework that utilizes execution overhead profiles to improve the efficiency of LLM-generated code. EffiLearner first generates code using an LLM, then executes it locally to capture execution time and memory usage profiles. These profiles are fed back to the LLM, which then revises the code to reduce overhead. To evaluate the effectiveness of EffiLearner, we conduct extensive experiments on the EffiBench, HumanEval, and MBPP with 16 open-source and 6 closed-source models. Our evaluation results demonstrate that through iterative self-optimization, EffiLearner significantly enhances the efficiency of LLM-generated code. For example, the execution time (ET) of StarCoder2-15B for the EffiBench decreases from 0.93 (s) to 0.12 (s) which reduces 87.1% the execution time requirement compared with the initial code. The total memory usage (TMU) of StarCoder2-15B also decreases from 22.02 (Mb*s) to 2.03 (Mb*s), which decreases 90.8% of total memory consumption during the execution process. The source code of EffiLearner was released in https://github.com/huangd1999/EffiLearner","url_abs":"https://arxiv.org/abs/2405.15189v4","url_pdf":"https://arxiv.org/pdf/2405.15189v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"soap-enhancing-efficiency-of-generated-code","repo_url":"https://github.com/huangd1999/effilearner","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"soap-enhancing-efficiency-of-generated-code","repo_url":"https://github.com/huangd1999/soap","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"soap-enhancing-efficiency-of-generated-code","repo_url":"https://github.com/yangyucheng000/University/tree/main/model-3/starcoder2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"humaneval","task_name":"HumanEval"},{"task_slug":null,"task_name":"mbpp"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.15189","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.15189"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/huangd1999/EffiLearner","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yangyucheng000/University/tree/main/model-3/starcoder2","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huangd1999/soap","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huangd1999/effilearner","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":10,"ran_draft_wrong":2,"unverified":5},"by_repo_kind":{"official":{"samples":17,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d55a48418c812b09","entry":"calculate_memory_usage","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/calculate_memory_usage.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/calculate_memory_usage.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d55a48418c812b09"}},{"code_sha256_prefix":"f21888a0452d23f3","entry":"calculate_memory_usage","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/code_efficiency_calculator.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/code_efficiency_calculator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f21888a0452d23f3"}},{"code_sha256_prefix":"6cf2449e22e03c59","entry":"calculate_runtime","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/calculate_memory_usage.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/calculate_memory_usage.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6cf2449e22e03c59"}},{"code_sha256_prefix":"952b9c9886c64735","entry":"calculate_runtime","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/code_efficiency_calculator.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/code_efficiency_calculator.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"952b9c9886c64735"}},{"code_sha256_prefix":"1cf1dc440f14d987","entry":"fetch_completion","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/gpt_EffiLearner.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/gpt_EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1cf1dc440f14d987"}},{"code_sha256_prefix":"a025fbf6131456dc","entry":"fetch_completion","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/gpt_generation.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/gpt_generation.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a025fbf6131456dc"}},{"code_sha256_prefix":"2c1a50ab9452b4aa","entry":"filter_line_profiler_results","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/run_source_code_five_times.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/run_source_code_five_times.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2c1a50ab9452b4aa"}},{"code_sha256_prefix":"b1ff21f11875705f","entry":"filter_memory_profiler_results","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/run_source_code_five_times.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/run_source_code_five_times.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b1ff21f11875705f"}},{"code_sha256_prefix":"4468d1d9afee5b05","entry":"prompt_construction","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/EffiLearner.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4468d1d9afee5b05"}},{"code_sha256_prefix":"09a23c4ef21bd633","entry":"prompt_construction","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/gpt_EffiLearner.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/gpt_EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"09a23c4ef21bd633"}},{"code_sha256_prefix":"1e930421d6b5e3e8","entry":"report_max_memory_usage","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/calculate_memory_usage.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/calculate_memory_usage.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1e930421d6b5e3e8"}},{"code_sha256_prefix":"41e149764f34c485","entry":"report_max_memory_usage","repo":"huangd1999/EffiLearner","repo_kind":"official","path":"src/code_efficiency_calculator.py","file_url":"https://github.com/huangd1999/EffiLearner/blob/HEAD/src/code_efficiency_calculator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41e149764f34c485"}},{"code_sha256_prefix":"2878904bfb0b29f6","entry":"construct_prompt_template","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/open_llm_generation.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/open_llm_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2878904bfb0b29f6"}},{"code_sha256_prefix":"59ed4ee229411304","entry":"construct_prompt_template","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/EffiLearner.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59ed4ee229411304"}},{"code_sha256_prefix":"71b8326ce231d069","entry":"fetch_completion","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/EffiLearner.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"71b8326ce231d069"}},{"code_sha256_prefix":"30c53b3e7b5e16e4","entry":"fetch_completion","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/open_llm_generation.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/open_llm_generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"30c53b3e7b5e16e4"}},{"code_sha256_prefix":"87352e224fad7dbd","entry":"fetch_completion","repo":"huangd1999/effilearner","repo_kind":"official","path":"src/EffiLearner.py","file_url":"https://github.com/huangd1999/effilearner/blob/HEAD/src/EffiLearner.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"87352e224fad7dbd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}