{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/omnicorpus-an-unified-multimodal-corpus-of-10","title":"OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text","arxiv_id":"2406.08418","date":"2024-06-12","proceeding":null,"authors":["Qingyun Li","Zhe Chen","Weiyun Wang","Wenhai Wang","Shenglong Ye","Zhenjiang Jin","Guanzhou Chen","Yinan He","Zhangwei Gao","Erfei Cui","Jiashuo Yu","Hao Tian","Jiasheng Zhou","Chao Xu","Bin Wang","Xingjian Wei","Wei Li","Wenjian Zhang","Bo Zhang","Pinlong Cai","Licheng Wen","Xiangchao Yan","Zhenxiang Li","Pei Chu","Yi Wang","Min Dou","Changyao Tian","Xizhou Zhu","Lewei Lu","Yushi Chen","Junjun He","Zhongying Tu","Tong Lu","Yali Wang","LiMin Wang","Dahua Lin","Yu Qiao","Botian Shi","Conghui He","Jifeng Dai"],"abstract":"Image-text interleaved data, consisting of multiple images and texts arranged in a natural document format, aligns with the presentation paradigm of internet data and closely resembles human reading habits. Recent studies have shown that such data aids multimodal in-context learning and maintains the capabilities of large language models during multimodal fine-tuning. However, the limited scale and diversity of current image-text interleaved data restrict the development of multimodal large language models. In this paper, we introduce OmniCorpus, a 10 billion-scale image-text interleaved dataset. Using an efficient data engine, we filter and extract large-scale high-quality documents, which contain 8.6 billion images and 1,696 billion text tokens. Compared to counterparts (e.g., MMC4, OBELICS), our dataset 1) has 15 times larger scales while maintaining good data quality; 2) features more diverse sources, including both English and non-English websites as well as video-centric websites; 3) is more flexible, easily degradable from an image-text interleaved format to pure text corpus and image-text pairs. Through comprehensive analysis and experiments, we validate the quality, usability, and effectiveness of the proposed dataset. We hope this could provide a solid data foundation for future multimodal model research. Code and data are released at https://github.com/OpenGVLab/OmniCorpus.","url_abs":"https://arxiv.org/abs/2406.08418v3","url_pdf":"https://arxiv.org/pdf/2406.08418v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"omnicorpus-an-unified-multimodal-corpus-of-10","repo_url":"https://github.com/opengvlab/omnicorpus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2406.08418","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.08418"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/opengvlab/omnicorpus","reach":{"status":"ok"}}],"summary":{"ran":2},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"2b3ea6be85a7e6cb","entry":"clean_attributes","repo":"opengvlab/omnicorpus","repo_kind":"official","path":"human_feedback_textfiltering/myxml.py","file_url":"https://github.com/opengvlab/omnicorpus/blob/HEAD/human_feedback_textfiltering/myxml.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b3ea6be85a7e6cb"}},{"code_sha256_prefix":"fc91265a186bcbd4","entry":"postprocess_captioning_generation","repo":"opengvlab/omnicorpus","repo_kind":"official","path":"mllm_internvl/coco_metric.py","file_url":"https://github.com/opengvlab/omnicorpus/blob/HEAD/mllm_internvl/coco_metric.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fc91265a186bcbd4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}