{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/doccoder-generating-code-by-retrieving-and","title":"DocPrompting: Generating Code by Retrieving the Docs","arxiv_id":"2207.05987","date":"2022-07-13","proceeding":null,"authors":["Shuyan Zhou","Uri Alon","Frank F. Xu","Zhiruo Wang","Zhengbao Jiang","Graham Neubig"],"abstract":"Publicly available source-code libraries are continuously growing and changing. This makes it impossible for models of code to keep current with all available APIs by simply training these models on existing code repositories. Thus, existing models inherently cannot generalize to using unseen functions and libraries, because these would never appear in the training data. In contrast, when human programmers use functions and libraries for the first time, they frequently refer to textual resources such as code manuals and documentation, to explore and understand the available functionality. Inspired by this observation, we introduce DocPrompting: a natural-language-to-code generation approach that explicitly leverages documentation by (1) retrieving the relevant documentation pieces given an NL intent, and (2) generating code based on the NL intent and the retrieved documentation. DocPrompting is general: it can be applied to any programming language and is agnostic to the underlying neural model. We demonstrate that DocPrompting consistently improves NL-to-code models: DocPrompting improves strong base models such as CodeT5 by 2.85% in pass@1 (52% relative gain) and 4.39% in pass@10 (30% relative gain) in execution-based evaluation on the popular Python CoNaLa benchmark; on a new Bash dataset tldr, DocPrompting improves CodeT5 and GPT-Neo1.3B by up to absolute 6.9% exact match.","url_abs":"https://arxiv.org/abs/2207.05987v3","url_pdf":"https://arxiv.org/pdf/2207.05987v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"doccoder-generating-code-by-retrieving-and","repo_url":"https://github.com/shuyanzhou/doccoder","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"doccoder-generating-code-by-retrieving-and","repo_url":"https://github.com/shuyanzhou/docprompting","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"}],"methods":[{"method_slug":"adafactor","method_name":"Adafactor"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"base","method_name":"BASE"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"glu","method_name":"Gated Linear Unit"},{"method_slug":"inverse-square-root-schedule","method_name":"Inverse Square Root Schedule"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sentencepiece","method_name":"SentencePiece"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"t5","method_name":"T5"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.05987","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.05987"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shuyanzhou/doccoder","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shuyanzhou/docprompting","reach":null}],"summary":{"ran":1,"ran_draft_wrong":2,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"951ed5b9657b55ce","entry":"RetrieverConfig","repo":"shuyanzhou/doccoder","repo_kind":"official","path":"generator/fid/src/model.py","file_url":"https://github.com/shuyanzhou/doccoder/blob/HEAD/generator/fid/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"951ed5b9657b55ce"}},{"code_sha256_prefix":"508a4a6a64754c3c","entry":"config","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/run_inference.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/run_inference.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"508a4a6a64754c3c"}},{"code_sha256_prefix":"37fe359be5cffce7","entry":"get_bert_embedding","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37fe359be5cffce7"}},{"code_sha256_prefix":"19c938f795d433a5","entry":"RetrievalModel","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19c938f795d433a5"}},{"code_sha256_prefix":"e6e89b2e0804678e","entry":"Retriever","repo":"shuyanzhou/doccoder","repo_kind":"official","path":"generator/fid/src/model.py","file_url":"https://github.com/shuyanzhou/doccoder/blob/HEAD/generator/fid/src/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6e89b2e0804678e"}},{"code_sha256_prefix":"9c210e01ad10a85d","entry":"bert_cos_score_idf","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c210e01ad10a85d"}},{"code_sha256_prefix":"fade9d8bfe4dbc38","entry":"collate_idf","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fade9d8bfe4dbc38"}},{"code_sha256_prefix":"38d2231884f32e2e","entry":"get_tokenizer","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"38d2231884f32e2e"}},{"code_sha256_prefix":"2c7a8491c88c3808","entry":"greedy_cos_idf_for_train","repo":"shuyanzhou/docprompting","repo_kind":"official","path":"retriever/simcse/model.py","file_url":"https://github.com/shuyanzhou/docprompting/blob/HEAD/retriever/simcse/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2c7a8491c88c3808"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}