{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/codehelp-using-large-language-models-with","title":"CodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes","arxiv_id":"2308.06921","date":"2023-08-14","proceeding":null,"authors":["Mark Liffiton","Brad Sheese","Jaromir Savelka","Paul Denny"],"abstract":"Computing educators face significant challenges in providing timely support to students, especially in large class settings. Large language models (LLMs) have emerged recently and show great promise for providing on-demand help at a large scale, but there are concerns that students may over-rely on the outputs produced by these models. In this paper, we introduce CodeHelp, a novel LLM-powered tool designed with guardrails to provide on-demand assistance to programming students without directly revealing solutions. We detail the design of the tool, which incorporates a number of useful features for instructors, and elaborate on the pipeline of prompting strategies we use to ensure generated outputs are suitable for students. To evaluate CodeHelp, we deployed it in a first-year computer and data science course with 52 students and collected student interactions over a 12-week period. We examine students' usage patterns and perceptions of the tool, and we report reflections from the course instructor and a series of recommendations for classroom use. Our findings suggest that CodeHelp is well-received by students who especially value its availability and help with resolving errors, and that for instructors it is easy to deploy and complements, rather than replaces, the support that they provide to students.","url_abs":"https://arxiv.org/abs/2308.06921v1","url_pdf":"https://arxiv.org/pdf/2308.06921v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"codehelp-using-large-language-models-with","repo_url":"https://github.com/liffiton/Gen-Ed/tree/main/src/codehelp","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"codehelp-using-large-language-models-with","repo_url":"https://github.com/liffiton/gen-ed","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.06921","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.06921"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liffiton/gen-ed","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liffiton/Gen-Ed/tree/main/src/codehelp","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"official":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"70368cc569c94a4c","entry":"get_completion","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/batch_llm.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/batch_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"70368cc569c94a4c"}},{"code_sha256_prefix":"608bc87f847cd41f","entry":"get_db","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/model_eval.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/model_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"608bc87f847cd41f"}},{"code_sha256_prefix":"b5d9156d92b9d114","entry":"model_string","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/loaders.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/loaders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5d9156d92b9d114"}},{"code_sha256_prefix":"a1e55a151ff6b775","entry":"msgs2str","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/query_tester.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/query_tester.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a1e55a151ff6b775"}},{"code_sha256_prefix":"65c4006eeb1a9e60","entry":"parse_model_string","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/loaders.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/loaders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"65c4006eeb1a9e60"}},{"code_sha256_prefix":"958b67913de0a015","entry":"process_batches","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/batch_llm.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/batch_llm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"958b67913de0a015"}},{"code_sha256_prefix":"fee03e1cc28353bd","entry":"check_one_access_control","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"src/gened/access.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/src/gened/access.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fee03e1cc28353bd"}},{"code_sha256_prefix":"210abdb74438bbda","entry":"choose_prompt_set","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"dev/model_eval.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/dev/model_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"210abdb74438bbda"}},{"code_sha256_prefix":"5ceb47d7a209c5e4","entry":"get_last_class","repo":"liffiton/Gen-Ed","repo_kind":"official","path":"src/gened/auth.py","file_url":"https://github.com/liffiton/Gen-Ed/blob/HEAD/src/gened/auth.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ceb47d7a209c5e4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}