{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fastfood-approximate-kernel-expansions-in","title":"Fastfood: Approximate Kernel Expansions in Loglinear Time","arxiv_id":"1408.3060","date":"2014-08-13","proceeding":null,"authors":["Quoc Viet Le","Tamas Sarlos","Alexander Johannes Smola"],"abstract":"Despite their successes, what makes kernel methods difficult to use in many\nlarge scale problems is the fact that storing and computing the decision\nfunction is typically expensive, especially at prediction time. In this paper,\nwe overcome this difficulty by proposing Fastfood, an approximation that\naccelerates such computation significantly. Key to Fastfood is the observation\nthat Hadamard matrices, when combined with diagonal Gaussian matrices, exhibit\nproperties similar to dense Gaussian random matrices. Yet unlike the latter,\nHadamard and diagonal matrices are inexpensive to multiply and store. These two\nmatrices can be used in lieu of Gaussian matrices in Random Kitchen Sinks\nproposed by Rahimi and Recht (2009) and thereby speeding up the computation for\na large range of kernel functions. Specifically, Fastfood requires O(n log d)\ntime and O(n) storage to compute n non-linear basis functions in d dimensions,\na significant improvement from O(nd) computation and storage, without\nsacrificing accuracy.\n  Our method applies to any translation invariant and any dot-product kernel,\nsuch as the popular RBF kernels and polynomial kernels. We prove that the\napproximation is unbiased and has low variance. Experiments show that we\nachieve similar accuracy to full kernel expansions and Random Kitchen Sinks\nwhile being 100x faster and using 1000x less memory. These improvements,\nespecially in terms of memory usage, make kernel methods more practical for\napplications that have large training sets and/or require real-time prediction.","url_abs":"http://arxiv.org/abs/1408.3060v1","url_pdf":"http://arxiv.org/pdf/1408.3060v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fastfood-approximate-kernel-expansions-in","repo_url":"https://github.com/AntoAndGar/Intrinsic-Dimension","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1408.3060","atlas_url":"https://app.syntology.ai/?focus=1408.3060","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}