{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/use-of-large-language-models-for-stance","title":"Prompting and Fine-Tuning Open-Sourced Large Language Models for Stance Classification","arxiv_id":"2309.13734","date":"2023-09-24","proceeding":null,"authors":["Iain J. Cruickshank","Lynnette Hui Xian Ng"],"abstract":"Stance classification, the task of predicting the viewpoint of an author on a subject of interest, has long been a focal point of research in domains ranging from social science to machine learning. Current stance detection methods rely predominantly on manual annotation of sentences, followed by training a supervised machine learning model. However, this manual annotation process requires laborious annotation effort, and thus hampers its potential to generalize across different contexts. In this work, we investigate the use of Large Language Models (LLMs) as a stance detection methodology that can reduce or even eliminate the need for manual annotations. We investigate 10 open-source models and 7 prompting schemes, finding that LLMs are competitive with in-domain supervised models but are not necessarily consistent in their performance. We also fine-tuned the LLMs, but discovered that fine-tuning process does not necessarily lead to better performance. In general, we discover that LLMs do not routinely outperform their smaller supervised machine learning models, and thus call for stance detection to be a benchmark for which LLMs also optimize for. The code used in this study is available at \\url{https://github.com/ijcruic/LLM-Stance-Labeling}","url_abs":"https://arxiv.org/abs/2309.13734v2","url_pdf":"https://arxiv.org/pdf/2309.13734v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"use-of-large-language-models-for-stance","repo_url":"https://github.com/ijcruic/llm-stance-labeling","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"stance-classification","task_name":"Stance Classification"},{"task_slug":"stance-detection","task_name":"Stance Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.13734","atlas_url":"https://app.syntology.ai/?focus=2309.13734","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.13734"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ijcruic/llm-stance-labeling","reach":{"status":"ok"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"63c1e5c6dc5be2f7","entry":"formatting_prompts_func","repo":"ijcruic/llm-stance-labeling","repo_kind":"official","path":"fine_tune_models.py","file_url":"https://github.com/ijcruic/llm-stance-labeling/blob/HEAD/fine_tune_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"63c1e5c6dc5be2f7"}},{"code_sha256_prefix":"a2639568066ec731","entry":"post_process_results","repo":"ijcruic/llm-stance-labeling","repo_kind":"official","path":"label_stance.py","file_url":"https://github.com/ijcruic/llm-stance-labeling/blob/HEAD/label_stance.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2639568066ec731"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}