{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/estimating-the-unseen-from-multiple","title":"Estimating the unseen from multiple populations","arxiv_id":"1707.03854","date":"2017-07-12","proceeding":"ICML 2017 8","authors":["Aditi Raghunathan","Greg Valiant","James Zou"],"abstract":"Given samples from a distribution, how many new elements should we expect to\nfind if we continue sampling this distribution? This is an important and\nactively studied problem, with many applications ranging from unseen species\nestimation to genomics. We generalize this extrapolation and related unseen\nestimation problems to the multiple population setting, where population $j$\nhas an unknown distribution $D_j$ from which we observe $n_j$ samples. We\nderive an optimal estimator for the total number of elements we expect to find\namong new samples across the populations. Surprisingly, we prove that our\nestimator's accuracy is independent of the number of populations. We also\ndevelop an efficient optimization algorithm to solve the more general problem\nof estimating multi-population frequency distributions. We validate our methods\nand theory through extensive experiments. Finally, on a real dataset of human\ngenomes across multiple ancestries, we demonstrate how our approach for unseen\nestimation can enable cohort designs that can discover interesting mutations\nwith greater efficiency.","url_abs":"http://arxiv.org/abs/1707.03854v1","url_pdf":"http://arxiv.org/pdf/1707.03854v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"estimating-the-unseen-from-multiple","repo_url":"https://github.com/roydeb/unseen_estimator","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"estimating-the-unseen-from-multiple","repo_url":"https://github.com/siddarthhari95/unseen_estimator","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}