{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-claude-3-model-family-opus-sonnet-haiku","title":"The Claude 3 Model Family: Opus, Sonnet, Haiku","arxiv_id":null,"date":"2024-03-04","proceeding":"Preprint 2024 3","authors":["Anthropic"],"abstract":"We introduce Claude 3, a new family of large multimodal models – Claude 3 Opus, our most capable offering, Claude 3 Sonnet, which provides a combination of skills and speed, and Claude 3 Haiku, our fastest and least expensive model. All new models have vision capabilities that enable them to process and analyze image data. The Claude 3 family demonstrates strong performance across benchmark evaluations and sets a new standard on measures of reasoning, math, and coding. Claude 3 Opus achieves state-of-the-art results on evaluations like GPQA [1], MMLU [2], MMMU [3] and many more. Claude 3 Haiku performs as well or better than Claude 2 [4] on most pure-text tasks, while Sonnet and Opus significantly outperform it. Additionally, these models exhibit improved fluency in non-English languages, making them more versatile for a global audience. In this report, we provide an in-depth analysis of our evaluations, focusing on core capabilities, safety, societal impacts, and the catastrophic risk assessments we committed to in our Responsible Scaling Policy.","url_abs":"https://www.anthropic.com/news/claude-3-family","url_pdf":"https://www-cdn.anthropic.com/de8ba9b01c9ab7cbabf5c33b80b7bbc618857627/Model_Card_Claude_3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"1-image-2-2-stitching","task_name":"1 Image, 2*2 Stitching"},{"task_slug":"arithmetic-reasoning","task_name":"Arithmetic Reasoning"},{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"common-sense-reasoning","task_name":"Common Sense Reasoning"},{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"long-context-understanding","task_name":"Long-Context Understanding"},{"task_slug":"mmlu","task_name":"MMLU"},{"task_slug":"math","task_name":"Math"},{"task_slug":"multi-task-language-understanding","task_name":"Multi-task Language Understanding"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"Claude 3 Opus (0-shot chain-of-thought)","rank_in_archive_order":8,"of":164,"metrics":{"Accuracy":"95"},"uses_additional_data":false},{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"Claude 3 Sonnet (0-shot chain-of-thought)","rank_in_archive_order":15,"of":164,"metrics":{"Accuracy":"92.3"},"uses_additional_data":false},{"leaderboard":"/sota/arithmetic-reasoning-on-gsm8k","task":"Arithmetic Reasoning","dataset":"GSM8K","model":"Claude 3 Haiku (0-shot chain-of-thought)","rank_in_archive_order":27,"of":164,"metrics":{"Accuracy":"88.9"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"Claude 3 Opus","rank_in_archive_order":11,"of":99,"metrics":{"Accuracy":"86.4"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"Claude 3 Haiku","rank_in_archive_order":22,"of":99,"metrics":{"Accuracy":"80.4"},"uses_additional_data":false},{"leaderboard":"/sota/code-generation-on-mbpp","task":"Code Generation","dataset":"MBPP","model":"Claude 3 Sonnet","rank_in_archive_order":25,"of":99,"metrics":{"Accuracy":"79.4"},"uses_additional_data":false},{"leaderboard":"/sota/common-sense-reasoning-on-winogrande","task":"Common Sense Reasoning","dataset":"WinoGrande","model":"Claude 3 Opus (5-shot)","rank_in_archive_order":6,"of":77,"metrics":{"Accuracy":"88.5"},"uses_additional_data":false},{"leaderboard":"/sota/common-sense-reasoning-on-winogrande","task":"Common Sense Reasoning","dataset":"WinoGrande","model":"Claude 3 Sonnet (5-shot)","rank_in_archive_order":27,"of":77,"metrics":{"Accuracy":"75.1"},"uses_additional_data":false},{"leaderboard":"/sota/common-sense-reasoning-on-winogrande","task":"Common Sense Reasoning","dataset":"WinoGrande","model":"Claude 3 Haiku (5-shot)","rank_in_archive_order":29,"of":77,"metrics":{"Accuracy":"74.2"},"uses_additional_data":false},{"leaderboard":"/sota/long-context-understanding-on-mmneedle","task":"Long-Context Understanding","dataset":"MMNeedle","model":"Claude 3 Opus","rank_in_archive_order":6,"of":12,"metrics":{"1 Image, 2*2 Stitching, Exact Accuracy":"52.25","1 Image, 4*4 Stitching, Exact Accuracy":"12.3","1 Image, 8*8 Stitching, Exact Accuracy":"1.6","10 Images, 1*1 Stitching, Exact Accuracy":"66.93","10 Images, 2*2 Stitching, Exact Accuracy":"4.6","10 Images, 4*4 Stitching, Exact Accuracy":"0.4","10 Images, 8*8 Stitching, Exact Accuracy":"0"},"uses_additional_data":false},{"leaderboard":"/sota/multi-task-language-understanding-on-mmlu","task":"Multi-task Language Understanding","dataset":"MML","model":"Claude 3 Sonnet (5-shot)","rank_in_archive_order":5,"of":44,"metrics":{"Average (%)":"79"},"uses_additional_data":false},{"leaderboard":"/sota/multi-task-language-understanding-on-mmlu","task":"Multi-task Language Understanding","dataset":"MML","model":"Claude 3 Haiku (5-shot)","rank_in_archive_order":7,"of":44,"metrics":{"Average (%)":"75.2"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-pubmedqa","task":"Question Answering","dataset":"PubMedQA","model":"Claude 3 Opus (5-shot)","rank_in_archive_order":15,"of":30,"metrics":{"Accuracy":"75.8"},"uses_additional_data":false},{"leaderboard":"/sota/question-answering-on-pubmedqa","task":"Question Answering","dataset":"PubMedQA","model":"Claude 3 Opus (zero-shot)","rank_in_archive_order":18,"of":30,"metrics":{"Accuracy":"74.9"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}