{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/focus-anywhere-for-fine-grained-multi-page","title":"Focus Anywhere for Fine-grained Multi-page Document Understanding","arxiv_id":"2405.14295","date":"2024-05-23","proceeding":null,"authors":["Chenglong Liu","Haoran Wei","Jinyue Chen","Lingyu Kong","Zheng Ge","Zining Zhu","Liang Zhao","Jianjian Sun","Chunrui Han","Xiangyu Zhang"],"abstract":"Modern LVLMs still struggle to achieve fine-grained document understanding, such as OCR/translation/caption for regions of interest to the user, tasks that require the context of the entire page, or even multiple pages. Accordingly, this paper proposes Fox, an effective pipeline, hybrid data, and tuning strategy, that catalyzes LVLMs to focus anywhere on single/multi-page documents. We introduce a novel task to boost the document understanding by making LVLMs focus attention on the document-level region, such as redefining full-page OCR as foreground focus. We employ multiple vision vocabularies to extract visual hybrid knowledge for interleaved document pages (e.g., a page containing a photo). Meanwhile, we render cross-vocabulary vision data as the catalyzer to achieve a full reaction of multiple visual vocabularies and in-document figure understanding. Further, without modifying the weights of multiple vision vocabularies, the above catalyzed fine-grained understanding capabilities can be efficiently tuned to multi-page documents, enabling the model to focus anywhere in both format-free and page-free manners. Besides, we build a benchmark including 9 fine-grained sub-tasks (e.g., region-level OCR/summary, color-guided OCR) to promote document analysis in the community. The experimental results verify the superiority of our model.","url_abs":"https://arxiv.org/abs/2405.14295v1","url_pdf":"https://arxiv.org/pdf/2405.14295v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"focus-anywhere-for-fine-grained-multi-page","repo_url":"https://github.com/Ucas-HaoranWei/Vary","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"optical-character-recognition","task_name":"Optical Character Recognition (OCR)"},{"task_slug":"document-understanding","task_name":"document understanding"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.14295","atlas_url":"https://app.syntology.ai/?focus=2405.14295","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14295"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Ucas-HaoranWei/Vary","reach":{"status":"ok"}}],"summary":{"ran":3,"ran_fixture":2,"ran_violates":1,"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"f1cb6f078273e8af","entry":"alb_wrapper","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/plug/transforms.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/plug/transforms.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1cb6f078273e8af"}},{"code_sha256_prefix":"293bca4e611ef1ce","entry":"autocontrast_func","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/plug/blip_process.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/plug/blip_process.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"293bca4e611ef1ce"}},{"code_sha256_prefix":"f5c4d23cd0722d0a","entry":"equalize_func","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/plug/blip_process.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/plug/blip_process.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f5c4d23cd0722d0a"}},{"code_sha256_prefix":"733d7f0bedcb74c2","entry":"get_rel_pos","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/vision_encoder/sam.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/vision_encoder/sam.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"733d7f0bedcb74c2"}},{"code_sha256_prefix":"1dd2ab205689d45b","entry":"identity_func","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/plug/blip_process.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/plug/blip_process.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1dd2ab205689d45b"}},{"code_sha256_prefix":"9b3c1cb391672ccb","entry":"load_image","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/demo/run_opt.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/demo/run_opt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9b3c1cb391672ccb"}},{"code_sha256_prefix":"105fa08885dc36cc","entry":"window_partition","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/vision_encoder/sam.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/vision_encoder/sam.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"105fa08885dc36cc"}},{"code_sha256_prefix":"27be441cc8213e52","entry":"window_unpartition","repo":"Ucas-HaoranWei/Vary","repo_kind":"listed","path":"Vary-master/vary/model/vision_encoder/sam.py","file_url":"https://github.com/Ucas-HaoranWei/Vary/blob/HEAD/Vary-master/vary/model/vision_encoder/sam.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"27be441cc8213e52"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}