{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/chinese-text-recognition-with-a-pre-trained","title":"Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning","arxiv_id":"2309.01083","date":"2023-09-03","proceeding":"ICCV 2023 1","authors":["Haiyang Yu","Xiaocong Wang","Bin Li","xiangyang xue"],"abstract":"Scene text recognition has been studied for decades due to its broad applications. However, despite Chinese characters possessing different characteristics from Latin characters, such as complex inner structures and large categories, few methods have been proposed for Chinese Text Recognition (CTR). Particularly, the characteristic of large categories poses challenges in dealing with zero-shot and few-shot Chinese characters. In this paper, inspired by the way humans recognize Chinese texts, we propose a two-stage framework for CTR. Firstly, we pre-train a CLIP-like model through aligning printed character images and Ideographic Description Sequences (IDS). This pre-training stage simulates humans recognizing Chinese characters and obtains the canonical representation of each character. Subsequently, the learned representations are employed to supervise the CTR model, such that traditional single-character recognition can be improved to text-line recognition through image-IDS matching. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on both Chinese character recognition (CCR) and CTR. The experimental results demonstrate that the proposed method performs best in CCR and outperforms previous methods in most scenarios of the CTR benchmark. It is worth noting that the proposed method can recognize zero-shot Chinese characters in text images without fine-tuning, whereas previous methods require fine-tuning when new classes appear. The code is available at https://github.com/FudanVI/FudanOCR/tree/main/image-ids-CTR.","url_abs":"https://arxiv.org/abs/2309.01083v1","url_pdf":"https://arxiv.org/pdf/2309.01083v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"chinese-text-recognition-with-a-pre-trained","repo_url":"https://github.com/fudanvi/fudanocr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"scene-text-recognition","task_name":"Scene Text Recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.01083","atlas_url":"https://app.syntology.ai/?focus=2309.01083","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.01083"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/FudanVI/FudanOCR","reach":null}],"summary":{"ran":4,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"3964a61b887ad29d","entry":"Bottleneck","repo":"FudanVI/FudanOCR","repo_kind":"official","path":"image-ids-CTR/CCR-CLIP/model.py","file_url":"https://github.com/FudanVI/FudanOCR/blob/HEAD/image-ids-CTR/CCR-CLIP/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3964a61b887ad29d"}},{"code_sha256_prefix":"4287507e69b4d25a","entry":"LayerNorm","repo":"FudanVI/FudanOCR","repo_kind":"official","path":"image-ids-CTR/CCR-CLIP/model.py","file_url":"https://github.com/FudanVI/FudanOCR/blob/HEAD/image-ids-CTR/CCR-CLIP/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4287507e69b4d25a"}},{"code_sha256_prefix":"177010bd20c75960","entry":"ResidualAttentionBlock","repo":"FudanVI/FudanOCR","repo_kind":"official","path":"image-ids-CTR/CCR-CLIP/model.py","file_url":"https://github.com/FudanVI/FudanOCR/blob/HEAD/image-ids-CTR/CCR-CLIP/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"177010bd20c75960"}},{"code_sha256_prefix":"692bca8496692c1f","entry":"Transformer","repo":"FudanVI/FudanOCR","repo_kind":"official","path":"image-ids-CTR/CCR-CLIP/model.py","file_url":"https://github.com/FudanVI/FudanOCR/blob/HEAD/image-ids-CTR/CCR-CLIP/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"692bca8496692c1f"}},{"code_sha256_prefix":"fe215b87ac8d71bf","entry":"CLIP","repo":"FudanVI/FudanOCR","repo_kind":"official","path":"image-ids-CTR/CCR-CLIP/model.py","file_url":"https://github.com/FudanVI/FudanOCR/blob/HEAD/image-ids-CTR/CCR-CLIP/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe215b87ac8d71bf"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}