{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/text-image-inpainting-via-global-structure","title":"Text Image Inpainting via Global Structure-Guided Diffusion Models","arxiv_id":"2401.14832","date":"2024-01-26","proceeding":null,"authors":["Shipeng Zhu","Pengfei Fang","Chenjie Zhu","Zuoyan Zhao","Qiang Xu","Hui Xue"],"abstract":"Real-world text can be damaged by corrosion issues caused by environmental or human factors, which hinder the preservation of the complete styles of texts, e.g., texture and structure. These corrosion issues, such as graffiti signs and incomplete signatures, bring difficulties in understanding the texts, thereby posing significant challenges to downstream applications, e.g., scene text recognition and signature identification. Notably, current inpainting techniques often fail to adequately address this problem and have difficulties restoring accurate text images along with reasonable and consistent styles. Formulating this as an open problem of text image inpainting, this paper aims to build a benchmark to facilitate its study. In doing so, we establish two specific text inpainting datasets which contain scene text images and handwritten text images, respectively. Each of them includes images revamped by real-life and synthetic datasets, featuring pairs of original images, corrupted images, and other assistant information. On top of the datasets, we further develop a novel neural framework, Global Structure-guided Diffusion Model (GSDM), as a potential solution. Leveraging the global structure of the text as a prior, the proposed GSDM develops an efficient diffusion model to recover clean texts. The efficacy of our approach is demonstrated by thorough empirical study, including a substantial boost in both recognition accuracy and image quality. These findings not only highlight the effectiveness of our method but also underscore its potential to enhance the broader field of text image understanding and processing. Code and datasets are available at: https://github.com/blackprotoss/GSDM.","url_abs":"https://arxiv.org/abs/2401.14832v3","url_pdf":"https://arxiv.org/pdf/2401.14832v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"text-image-inpainting-via-global-structure","repo_url":"https://github.com/blackprotoss/gsdm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"text-image-inpainting-via-global-structure","repo_url":"https://github.com/jdfxzzy/pean","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-inpainting","task_name":"Image Inpainting"},{"task_slug":"scene-text-recognition","task_name":"Scene Text Recognition"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"pixel-prediction","method_name":"Inpainting"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2401.14832","atlas_url":"https://app.syntology.ai/?focus=2401.14832","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2401.14832"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/blackprotoss/GSDM","reach":null}],"summary":{"ran":4,"unverified":7},"by_repo_kind":{"official":{"samples":11,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ae0ed693b569f627","entry":"BaseModel","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ae0ed693b569f627"}},{"code_sha256_prefix":"17605665a45a8b50","entry":"Conv_bn_block","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17605665a45a8b50"}},{"code_sha256_prefix":"80e7e021706c3f2b","entry":"Res_block","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80e7e021706c3f2b"}},{"code_sha256_prefix":"6c3d3d8b4a1de911","entry":"SPM","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c3d3d8b4a1de911"}},{"code_sha256_prefix":"ec6180301c84d572","entry":"DDPM","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec6180301c84d572"}},{"code_sha256_prefix":"8be0463898a217a2","entry":"GSDM","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8be0463898a217a2"}},{"code_sha256_prefix":"92dfbdb3fa59745e","entry":"define_G","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92dfbdb3fa59745e"}},{"code_sha256_prefix":"3d9b1d0b93e94962","entry":"init_weights","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d9b1d0b93e94962"}},{"code_sha256_prefix":"26ac90bf0df6c22e","entry":"weights_init_kaiming","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26ac90bf0df6c22e"}},{"code_sha256_prefix":"e458d481e772f2a0","entry":"weights_init_normal","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e458d481e772f2a0"}},{"code_sha256_prefix":"b61e63d89ec6d881","entry":"weights_init_orthogonal","repo":"blackprotoss/GSDM","repo_kind":"official","path":"model/model.py","file_url":"https://github.com/blackprotoss/GSDM/blob/HEAD/model/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b61e63d89ec6d881"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}