{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/simpledoc-multi-modal-document-understanding","title":"SimpleDoc: Multi-Modal Document Understanding with Dual-Cue Page Retrieval and Iterative Refinement","arxiv_id":"2506.14035","date":"2025-06-16","proceeding":null,"authors":["Chelsi Jain","Yiran Wu","Yifan Zeng","Jiale Liu","S hengyu Dai","Zhenwen Shao","Qingyun Wu","Huazheng Wang"],"abstract":"Document Visual Question Answering (DocVQA) is a practical yet challenging task, which is to ask questions based on documents while referring to multiple pages and different modalities of information, e.g, images and tables. To handle multi-modality, recent methods follow a similar Retrieval Augmented Generation (RAG) pipeline, but utilize Visual Language Models (VLMs) based embedding model to embed and retrieve relevant pages as images, and generate answers with VLMs that can accept an image as input. In this paper, we introduce SimpleDoc, a lightweight yet powerful retrieval - augmented framework for DocVQA. It boosts evidence page gathering by first retrieving candidates through embedding similarity and then filtering and re-ranking these candidates based on page summaries. A single VLM-based reasoner agent repeatedly invokes this dual-cue retriever, iteratively pulling fresh pages into a working memory until the question is confidently answered. SimpleDoc outperforms previous baselines by 3.2% on average on 4 DocVQA datasets with much fewer pages retrieved. Our code is available at https://github.com/ag2ai/SimpleDoc.","url_abs":"https://arxiv.org/abs/2506.14035v1","url_pdf":"https://arxiv.org/pdf/2506.14035v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"simpledoc-multi-modal-document-understanding","repo_url":"https://github.com/ag2ai/simpledoc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"re-ranking","task_name":"Re-Ranking"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"document-understanding","task_name":"document understanding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.14035","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.14035"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ag2ai/SimpleDoc","reach":{"status":"ok"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"36d783e0916c23cb","entry":"calculate_metrics","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/analyze_multiple_runs.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/analyze_multiple_runs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"36d783e0916c23cb"}},{"code_sha256_prefix":"d41c5a9581a7862d","entry":"evaluate_response","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/evaluate_responses.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/evaluate_responses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d41c5a9581a7862d"}},{"code_sha256_prefix":"442a4d51b3fdf933","entry":"extract_document_summary","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"modules/step02_page_retrieval.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/modules/step02_page_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"442a4d51b3fdf933"}},{"code_sha256_prefix":"dcb682866c231e85","entry":"extract_relevant_pages","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"modules/step02_page_retrieval.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/modules/step02_page_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dcb682866c231e85"}},{"code_sha256_prefix":"0017500466774ab3","entry":"is_ground_truth_not_answerable","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/evaluate_pipeline.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/evaluate_pipeline.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0017500466774ab3"}},{"code_sha256_prefix":"318f11f47eb00c0c","entry":"load_eval_results","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/analyze_multiple_runs.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/analyze_multiple_runs.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"318f11f47eb00c0c"}},{"code_sha256_prefix":"a60837f27b20ddec","entry":"load_ground_truth","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/evaluate_responses.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/evaluate_responses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a60837f27b20ddec"}},{"code_sha256_prefix":"d96da92d8a935925","entry":"load_results","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"evaluation/evaluate_responses.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/evaluation/evaluate_responses.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d96da92d8a935925"}},{"code_sha256_prefix":"e9fd0d235482a280","entry":"postprocess_answer","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"modules/step03_target_page_qa.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/modules/step03_target_page_qa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e9fd0d235482a280"}},{"code_sha256_prefix":"b75b739aeeeb160e","entry":"query_llm_for_page_retrieval","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"modules/step02_page_retrieval.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/modules/step02_page_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b75b739aeeeb160e"}},{"code_sha256_prefix":"b95b72231eb3aa73","entry":"query_llm_with_images","repo":"ag2ai/SimpleDoc","repo_kind":"official","path":"modules/step03_target_page_qa.py","file_url":"https://github.com/ag2ai/SimpleDoc/blob/HEAD/modules/step03_target_page_qa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b95b72231eb3aa73"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}