{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/koala-an-index-for-quantifying-overlaps-with","title":"Koala: An Index for Quantifying Overlaps with Pre-training Corpora","arxiv_id":"2303.14770","date":"2023-03-26","proceeding":null,"authors":["Thuy-Trang Vu","Xuanli He","Gholamreza Haffari","Ehsan Shareghi"],"abstract":"In very recent years more attention has been placed on probing the role of pre-training data in Large Language Models (LLMs) downstream behaviour. Despite the importance, there is no public tool that supports such analysis of pre-training corpora at large scale. To help research in this space, we launch Koala, a searchable index over large pre-training corpora using compressed suffix arrays with highly efficient compression rate and search support. In its first release we index the public proportion of OPT 175B pre-training data. Koala provides a framework to do forensic analysis on the current and future benchmarks as well as to assess the degree of memorization in the output from the LLMs. Koala is available for public use at https://koala-index.erc.monash.edu/.","url_abs":"https://arxiv.org/abs/2303.14770v1","url_pdf":"https://arxiv.org/pdf/2303.14770v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"memorization","task_name":"Memorization"}],"methods":[{"method_slug":"opt","method_name":"OPT"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2303.14770","atlas_url":"https://app.syntology.ai/?focus=2303.14770","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2303.14770"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ekzhu/datasketch","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bd7d7a9c3a6709ee","entry":"optimal_partitions","repo":"ekzhu/datasketch","repo_kind":"found_in_text","path":"datasketch/lshensemble_partition.py","file_url":"https://github.com/ekzhu/datasketch/blob/HEAD/datasketch/lshensemble_partition.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd7d7a9c3a6709ee"}},{"code_sha256_prefix":"6dc9da353c1b85b0","entry":"sha1_hash32","repo":"ekzhu/datasketch","repo_kind":"found_in_text","path":"datasketch/hashfunc.py","file_url":"https://github.com/ekzhu/datasketch/blob/HEAD/datasketch/hashfunc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6dc9da353c1b85b0"}},{"code_sha256_prefix":"9c25f231de344715","entry":"sha1_hash64","repo":"ekzhu/datasketch","repo_kind":"found_in_text","path":"datasketch/hashfunc.py","file_url":"https://github.com/ekzhu/datasketch/blob/HEAD/datasketch/hashfunc.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9c25f231de344715"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}