{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/maximally-consistent-sampling-and-the-jaccard","title":"Maximally Consistent Sampling and the Jaccard Index of Probability Distributions","arxiv_id":"1809.04052","date":"2018-10-24","proceeding":null,"authors":["Moulton Ryan","Jiang Yunjiang"],"abstract":"We introduce simple, efficient algorithms for computing a MinHash of a\nprobability distribution, suitable for both sparse and dense data, with\nequivalent running times to the state of the art for both cases. The collision\nprobability of these algorithms is a new measure of the similarity of positive\nvectors which we investigate in detail. We describe the sense in which this\ncollision probability is optimal for any Locality Sensitive Hash based on\nsampling. We argue that this similarity measure is more useful for probability\ndistributions than the similarity pursued by other algorithms for weighted\nMinHash, and is the natural generalization of the Jaccard index.","url_abs":"http://arxiv.org/abs/1809.04052v2","url_pdf":"http://arxiv.org/pdf/1809.04052v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"maximally-consistent-sampling-and-the-jaccard","repo_url":"https://github.com/jean-pierreBoth/probminhash","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.04052","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}