{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/prompting-with-pseudo-code-instructions","title":"Prompting with Pseudo-Code Instructions","arxiv_id":"2305.11790","date":"2023-05-19","proceeding":null,"authors":["Mayank Mishra","Prince Kumar","Riyaz Bhat","Rudra Murthy V","Danish Contractor","Srikanth Tamilselvam"],"abstract":"Prompting with natural language instructions has recently emerged as a popular method of harnessing the capabilities of large language models. Given the inherent ambiguity present in natural language, it is intuitive to consider the possible advantages of prompting with less ambiguous prompt styles, such as the use of pseudo-code. In this paper we explore if prompting via pseudo-code instructions helps improve the performance of pre-trained language models. We manually create a dataset of pseudo-code prompts for 132 different tasks spanning classification, QA and generative language tasks, sourced from the Super-NaturalInstructions dataset. Using these prompts along with their counterparts in natural language, we study their performance on two LLM families - BLOOM and CodeGen. Our experiments show that using pseudo-code instructions leads to better results, with an average increase (absolute) of 7-16 points in F1 scores for classification tasks and an improvement (relative) of 12-38% in aggregate ROUGE-L scores across all tasks. We include detailed ablation studies which indicate that code comments, docstrings, and the structural clues encoded in pseudo-code all contribute towards the improvement in performance. To the best of our knowledge our work is the first to demonstrate how pseudo-code prompts can be helpful in improving the performance of pre-trained LMs.","url_abs":"https://arxiv.org/abs/2305.11790v3","url_pdf":"https://arxiv.org/pdf/2305.11790v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"prompting-with-pseudo-code-instructions","repo_url":"https://github.com/mayank31398/pseudo-code-instructions","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"prompting-with-pseudo-code-instructions","repo_url":"https://github.com/paralleldrive/sudolang-llm-support","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"bloom","method_name":"BLOOM"},{"method_slug":"codegen","method_name":"CodeGen"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2305.11790","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.11790"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/paralleldrive/sudolang-llm-support","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mayank31398/pseudo-code-instructions","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":13},"by_repo_kind":{"official":{"samples":13,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cea75406d359ed14","entry":"english_to_gujarati","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1079.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1079.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cea75406d359ed14"}},{"code_sha256_prefix":"35ab7eb005ae0a76","entry":"load_tokenizer","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"code_instruct/model.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/code_instruct/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35ab7eb005ae0a76"}},{"code_sha256_prefix":"df1b3e1970a39db7","entry":"pad","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"code_instruct/model.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/code_instruct/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"df1b3e1970a39db7"}},{"code_sha256_prefix":"b8d1dd9b82ebdfde","entry":"preprocess","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1007.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1007.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b8d1dd9b82ebdfde"}},{"code_sha256_prefix":"ba93e42d23bea00f","entry":"preprocess","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1019.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1019.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba93e42d23bea00f"}},{"code_sha256_prefix":"43f4d8361d5f4aee","entry":"preprocess","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/105.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/105.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43f4d8361d5f4aee"}},{"code_sha256_prefix":"54a724b27bcc9357","entry":"preprocess","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1061.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1061.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"54a724b27bcc9357"}},{"code_sha256_prefix":"d493bce6bf95b3e1","entry":"sort","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"code_instruct/split_tasks.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/code_instruct/split_tasks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d493bce6bf95b3e1"}},{"code_sha256_prefix":"08da323c868e03b6","entry":"story_sentence_generation","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/105.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/105.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"08da323c868e03b6"}},{"code_sha256_prefix":"dfa75eb02881dee8","entry":"translate_bengali_to_marathi","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1061.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1061.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dfa75eb02881dee8"}},{"code_sha256_prefix":"124a5b44b23a83b5","entry":"translate_english_to_punjabi","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1007.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1007.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"124a5b44b23a83b5"}},{"code_sha256_prefix":"71ac44a0094c857b","entry":"translate_oriya_to_tamil","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1073.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1073.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71ac44a0094c857b"}},{"code_sha256_prefix":"aa32f3d974f6de21","entry":"translate_oriya_to_telugu","repo":"mayank31398/pseudo-code-instructions","repo_kind":"official","path":"instructions/1019.py","file_url":"https://github.com/mayank31398/pseudo-code-instructions/blob/HEAD/instructions/1019.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa32f3d974f6de21"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}