{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/megaman-manifold-learning-with-millions-of","title":"megaman: Manifold Learning with Millions of points","arxiv_id":"1603.02763","date":"2016-03-09","proceeding":null,"authors":["James McQueen","Marina Meila","Jacob VanderPlas","Zhongyue Zhang"],"abstract":"Manifold Learning is a class of algorithms seeking a low-dimensional\nnon-linear representation of high-dimensional data. Thus manifold learning\nalgorithms are, at least in theory, most applicable to high-dimensional data\nand sample sizes to enable accurate estimation of the manifold. Despite this,\nmost existing manifold learning implementations are not particularly scalable.\nHere we present a Python package that implements a variety of manifold learning\nalgorithms in a modular and scalable fashion, using fast approximate neighbors\nsearches and fast sparse eigendecompositions. The package incorporates\ntheoretical advances in manifold learning, such as the unbiased Laplacian\nestimator and the estimation of the embedding distortion by the Riemannian\nmetric method. In benchmarks, even on a single-core desktop computer, our code\nembeds millions of data points in minutes, and takes just 200 minutes to embed\nthe main sample of galaxy spectra from the Sloan Digital Sky Survey ---\nconsisting of 0.6 million samples in 3750-dimensions --- a task which has not\npreviously been possible.","url_abs":"http://arxiv.org/abs/1603.02763v1","url_pdf":"http://arxiv.org/pdf/1603.02763v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"megaman-manifold-learning-with-millions-of","repo_url":"https://github.com/mmp2/megaman","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}