{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/codegeex-a-pre-trained-model-for-code","title":"CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X","arxiv_id":"2303.17568","date":"2023-03-30","proceeding":null,"authors":["Qinkai Zheng","Xiao Xia","Xu Zou","Yuxiao Dong","Shan Wang","Yufei Xue","Zihan Wang","Lei Shen","Andi Wang","Yang Li","Teng Su","Zhilin Yang","Jie Tang"],"abstract":"Large pre-trained code generation models, such as OpenAI Codex, can generate syntax- and function-correct code, making the coding of programmers more productive and our pursuit of artificial general intelligence closer. In this paper, we introduce CodeGeeX, a multilingual model with 13 billion parameters for code generation. CodeGeeX is pre-trained on 850 billion tokens of 23 programming languages as of June 2022. Our extensive experiments suggest that CodeGeeX outperforms multilingual code models of similar scale for both the tasks of code generation and translation on HumanEval-X. Building upon HumanEval (Python only), we develop the HumanEval-X benchmark for evaluating multilingual models by hand-writing the solutions in C++, Java, JavaScript, and Go. In addition, we build CodeGeeX-based extensions on Visual Studio Code, JetBrains, and Cloud Studio, generating 4.7 billion tokens for tens of thousands of active users per week. Our user study demonstrates that CodeGeeX can help to increase coding efficiency for 83.4% of its users. Finally, CodeGeeX is publicly accessible and in Sep. 2022, we open-sourced its code, model weights (the version of 850B tokens), API, extensions, and HumanEval-X at https://github.com/THUDM/CodeGeeX.","url_abs":"https://arxiv.org/abs/2303.17568v2","url_pdf":"https://arxiv.org/pdf/2303.17568v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"codegeex-a-pre-trained-model-for-code","repo_url":"https://github.com/THUDM/CodeGeeX","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"mindspore","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"codegeex-a-pre-trained-model-for-code","repo_url":"https://github.com/thudm/codegeex2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"humaneval","task_name":"HumanEval"}],"methods":[],"datasets_introduced":[{"slug":"humaneval-x","name":"HumanEval-X","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"CodeGeeX-13B","rank_in_archive_order":95,"of":99,"metrics":{"Accuracy":"24.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2303.17568","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.17568"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thudm/codegeex2","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/THUDM/CodeGeeX","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"unverified":6},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1},"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2a6537281349742d","entry":"fast_gelu","repo":"THUDM/CodeGeeX","repo_kind":"official","path":"codegeex/torch/codegeex_model.py","file_url":"https://github.com/THUDM/CodeGeeX/blob/HEAD/codegeex/torch/codegeex_model.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2a6537281349742d"}},{"code_sha256_prefix":"f12f9486ba2960e6","entry":"set_weight_decay","repo":"THUDM/CodeGeeX","repo_kind":"official","path":"codegeex/mindspore/finetune.py","file_url":"https://github.com/THUDM/CodeGeeX/blob/HEAD/codegeex/mindspore/finetune.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f12f9486ba2960e6"}},{"code_sha256_prefix":"42a53b2c340cdf87","entry":"add_code_generation_args","repo":"thudm/codegeex2","repo_kind":"listed","path":"demo/fastapicpu.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/demo/fastapicpu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"42a53b2c340cdf87"}},{"code_sha256_prefix":"c25730f1da3ae813","entry":"add_code_generation_args","repo":"thudm/codegeex2","repo_kind":"listed","path":"demo/run_demo.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/demo/run_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c25730f1da3ae813"}},{"code_sha256_prefix":"cd91456d18781855","entry":"auto_configure_device_map","repo":"thudm/codegeex2","repo_kind":"listed","path":"demo/gpus.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/demo/gpus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cd91456d18781855"}},{"code_sha256_prefix":"b5f56a90751fcf4d","entry":"get_model","repo":"thudm/codegeex2","repo_kind":"listed","path":"demo/run_demo.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/demo/run_demo.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b5f56a90751fcf4d"}},{"code_sha256_prefix":"eb072eed78a57ca7","entry":"load_model_on_gpus","repo":"thudm/codegeex2","repo_kind":"listed","path":"demo/gpus.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/demo/gpus.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eb072eed78a57ca7"}},{"code_sha256_prefix":"637b4d55eb53b273","entry":"postprocess_generation","repo":"thudm/codegeex2","repo_kind":"listed","path":"evaluation/evaluation.py","file_url":"https://github.com/thudm/codegeex2/blob/HEAD/evaluation/evaluation.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"637b4d55eb53b273"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}