{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-new-unbiased-and-efficient-class-of-lsh","title":"A New Unbiased and Efficient Class of LSH-Based Samplers and Estimators for Partition Function Computation in Log-Linear Models","arxiv_id":"1703.05160","date":"2017-03-15","proceeding":null,"authors":["Ryan Spring","Anshumali Shrivastava"],"abstract":"Log-linear models are arguably the most successful class of graphical models\nfor large-scale applications because of their simplicity and tractability.\nLearning and inference with these models require calculating the partition\nfunction, which is a major bottleneck and intractable for large state spaces.\nImportance Sampling (IS) and MCMC-based approaches are lucrative. However, the\ncondition of having a \"good\" proposal distribution is often not satisfied in\npractice.\n  In this paper, we add a new dimension to efficient estimation via sampling.\nWe propose a new sampling scheme and an unbiased estimator that estimates the\npartition function accurately in sub-linear time. Our samples are generated in\nnear-constant time using locality sensitive hashing (LSH), and so are\ncorrelated and unnormalized. We demonstrate the effectiveness of our proposed\napproach by comparing the accuracy and speed of estimating the partition\nfunction against other state-of-the-art estimation techniques including IS and\nthe efficient variant of Gumbel-Max sampling. With our efficient sampling\nscheme, we accurately train real-world language models using only 1-2% of\ncomputations.","url_abs":"http://arxiv.org/abs/1703.05160v1","url_pdf":"http://arxiv.org/pdf/1703.05160v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-new-unbiased-and-efficient-class-of-lsh","repo_url":"https://github.com/rdspring1/lsh_deeplearning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1703.05160","atlas_url":"https://app.syntology.ai/?focus=1703.05160","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}