{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-ocr-capabilities-of-gpt-4v-ision-a","title":"Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation","arxiv_id":"2310.16809","date":"2023-10-25","proceeding":null,"authors":["Yongxin Shi","Dezhi Peng","Wenhui Liao","Zening Lin","Xinhong Chen","Chongyu Liu","Yuyi Zhang","Lianwen Jin"],"abstract":"This paper presents a comprehensive evaluation of the Optical Character Recognition (OCR) capabilities of the recently released GPT-4V(ision), a Large Multimodal Model (LMM). We assess the model's performance across a range of OCR tasks, including scene text recognition, handwritten text recognition, handwritten mathematical expression recognition, table structure recognition, and information extraction from visually-rich document. The evaluation reveals that GPT-4V performs well in recognizing and understanding Latin contents, but struggles with multilingual scenarios and complex tasks. Specifically, it showed limitations when dealing with non-Latin languages and complex tasks such as handwriting mathematical expression recognition, table structure recognition, and end-to-end semantic entity recognition and pair extraction from document image. Based on these observations, we affirm the necessity and continued research value of specialized OCR models. In general, despite its versatility in handling diverse OCR tasks, GPT-4V does not outperform existing state-of-the-art OCR models. How to fully utilize pre-trained general-purpose LMMs such as GPT-4V for OCR downstream tasks remains an open problem. The study offers a critical reference for future research in OCR with LMMs. Evaluation pipeline and results are available at https://github.com/SCUT-DLVCLab/GPT-4V_OCR.","url_abs":"https://arxiv.org/abs/2310.16809v2","url_pdf":"https://arxiv.org/pdf/2310.16809v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-ocr-capabilities-of-gpt-4v-ision-a","repo_url":"https://github.com/scut-dlvclab/gpt-4v_ocr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"handwritten-text-recognition","task_name":"Handwritten Text Recognition"},{"task_slug":"key-information-extraction","task_name":"Key Information Extraction"},{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition"},{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition (OCR)"},{"task_slug":"scene-text-recognition","task_name":"Scene Text Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2310.16809","atlas_url":"https://app.syntology.ai/?focus=2310.16809","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.16809"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scut-dlvclab/gpt-4v_ocr","reach":null}],"summary":{"ran_draft_wrong":3,"ran_violates":1},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"edcdb7863ec0dfad","entry":"char_f2h","repo":"scut-dlvclab/gpt-4v_ocr","repo_kind":"official","path":"src/common/string_utils.py","file_url":"https://github.com/scut-dlvclab/gpt-4v_ocr/blob/HEAD/src/common/string_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"edcdb7863ec0dfad"}},{"code_sha256_prefix":"71017b83f2989c5f","entry":"is_chinese","repo":"scut-dlvclab/gpt-4v_ocr","repo_kind":"official","path":"src/common/string_utils.py","file_url":"https://github.com/scut-dlvclab/gpt-4v_ocr/blob/HEAD/src/common/string_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"71017b83f2989c5f"}},{"code_sha256_prefix":"1c139d54f78c0377","entry":"split_token","repo":"scut-dlvclab/gpt-4v_ocr","repo_kind":"official","path":"src/hmer/eval_HMER.py","file_url":"https://github.com/scut-dlvclab/gpt-4v_ocr/blob/HEAD/src/hmer/eval_HMER.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1c139d54f78c0377"}},{"code_sha256_prefix":"9caa37e59011da50","entry":"string_f2h","repo":"scut-dlvclab/gpt-4v_ocr","repo_kind":"official","path":"src/common/string_utils.py","file_url":"https://github.com/scut-dlvclab/gpt-4v_ocr/blob/HEAD/src/common/string_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9caa37e59011da50"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}