{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sentiment-analysis-in-the-era-of-large","title":"Sentiment Analysis in the Era of Large Language Models: A Reality Check","arxiv_id":"2305.15005","date":"2023-05-24","proceeding":null,"authors":["Wenxuan Zhang","Yue Deng","Bing Liu","Sinno Jialin Pan","Lidong Bing"],"abstract":"Sentiment analysis (SA) has been a long-standing research area in natural language processing. It can offer rich insights into human sentiments and opinions and has thus seen considerable interest from both academia and industry. With the advent of large language models (LLMs) such as ChatGPT, there is a great potential for their employment on SA problems. However, the extent to which existing LLMs can be leveraged for different sentiment analysis tasks remains unclear. This paper aims to provide a comprehensive investigation into the capabilities of LLMs in performing various sentiment analysis tasks, from conventional sentiment classification to aspect-based sentiment analysis and multifaceted analysis of subjective texts. We evaluate performance across 13 tasks on 26 datasets and compare the results against small language models (SLMs) trained on domain-specific datasets. Our study reveals that while LLMs demonstrate satisfactory performance in simpler tasks, they lag behind in more complex tasks requiring deeper understanding or structured sentiment information. However, LLMs significantly outperform SLMs in few-shot learning settings, suggesting their potential when annotation resources are limited. We also highlight the limitations of current evaluation practices in assessing LLMs' SA abilities and propose a novel benchmark, \\textsc{SentiEval}, for a more comprehensive and realistic evaluation. Data and code during our investigations are available at \\url{https://github.com/DAMO-NLP-SG/LLM-Sentiment}.","url_abs":"https://arxiv.org/abs/2305.15005v1","url_pdf":"https://arxiv.org/pdf/2305.15005v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sentiment-analysis-in-the-era-of-large","repo_url":"https://github.com/damo-nlp-sg/llm-sentiment","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"aspect-based-sentiment-analysis-1","task_name":"Aspect-Based Sentiment Analysis"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"sentiment-classification","task_name":"Sentiment Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2305.15005","atlas_url":"https://app.syntology.ai/?focus=2305.15005","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.15005"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/damo-nlp-sg/llm-sentiment","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"10ef73cfeb03a631","entry":"extract_label","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"10ef73cfeb03a631"}},{"code_sha256_prefix":"3ed2c1a495206991","entry":"extract_labels","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"3ed2c1a495206991"}},{"code_sha256_prefix":"228207619251d2ad","entry":"parallel_query_chatgpt_model","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"predict.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/predict.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"228207619251d2ad"}},{"code_sha256_prefix":"406e5c992288f1dd","entry":"parallel_query_davinci_model","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"predict.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/predict.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"406e5c992288f1dd"}},{"code_sha256_prefix":"5da8031f78e9aa57","entry":"process_tuple_f1","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"evaluate.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/evaluate.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"5da8031f78e9aa57"}},{"code_sha256_prefix":"aa9f51571e5694f7","entry":"query_chatgpt_model","repo":"damo-nlp-sg/llm-sentiment","repo_kind":"official","path":"predict.py","file_url":"https://github.com/damo-nlp-sg/llm-sentiment/blob/HEAD/predict.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"aa9f51571e5694f7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}