{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/promptsource-an-integrated-development","title":"PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts","arxiv_id":"2202.01279","date":"2022-02-02","proceeding":"ACL 2022 5","authors":["Stephen H. Bach","Victor Sanh","Zheng-Xin Yong","Albert Webson","Colin Raffel","Nihal V. Nayak","Abheesht Sharma","Taewoon Kim","M Saiful Bari","Thibault Fevry","Zaid Alyafeai","Manan Dey","Andrea Santilli","Zhiqing Sun","Srulik Ben-David","Canwen Xu","Gunjan Chhablani","Han Wang","Jason Alan Fries","Maged S. Al-shaibani","Shanya Sharma","Urmish Thakker","Khalid Almubarak","Xiangru Tang","Dragomir Radev","Mike Tian-Jian Jiang","Alexander M. Rush"],"abstract":"PromptSource is a system for creating, sharing, and using natural language prompts. Prompts are functions that map an example from a dataset to a natural language input and target output. Using prompts to train and query language models is an emerging area in NLP that requires new tools that let users develop and refine these prompts collaboratively. PromptSource addresses the emergent challenges in this new setting with (1) a templating language for defining data-linked prompts, (2) an interface that lets users quickly iterate on prompt development by observing outputs of their prompts on many examples, and (3) a community-driven set of guidelines for contributing new prompts to a common pool. Over 2,000 prompts for roughly 170 datasets are already available in PromptSource. PromptSource is available at https://github.com/bigscience-workshop/promptsource.","url_abs":"https://arxiv.org/abs/2202.01279v3","url_pdf":"https://arxiv.org/pdf/2202.01279v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"promptsource-an-integrated-development","repo_url":"https://github.com/bigscience-workshop/promptsource","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.01279","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.01279"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bigscience-workshop/promptsource","reach":null}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"83814e79cf5e7fe0","entry":"Template","repo":"bigscience-workshop/promptsource","repo_kind":"official","path":"promptsource/templates.py","file_url":"https://github.com/bigscience-workshop/promptsource/blob/HEAD/promptsource/templates.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"83814e79cf5e7fe0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}