{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-random-binning-features-fast","title":"Revisiting Random Binning Features: Fast Convergence and Strong Parallelizability","arxiv_id":"1809.05247","date":"2018-09-14","proceeding":null,"authors":["Lingfei Wu","Ian E. H. Yen","Jie Chen","Rui Yan"],"abstract":"Kernel method has been developed as one of the standard approaches for\nnonlinear learning, which however, does not scale to large data set due to its\nquadratic complexity in the number of samples. A number of kernel approximation\nmethods have thus been proposed in the recent years, among which the random\nfeatures method gains much popularity due to its simplicity and direct\nreduction of nonlinear problem to a linear one. The Random Binning (RB)\nfeature, proposed in the first random-feature paper \\cite{rahimi2007random},\nhas drawn much less attention than the Random Fourier (RF) feature. In this\nwork, we observe that the RB features, with right choice of optimization\nsolver, could be orders-of-magnitude more efficient than other random features\nand kernel approximation methods under the same requirement of accuracy. We\nthus propose the first analysis of RB from the perspective of optimization,\nwhich by interpreting RB as a Randomized Block Coordinate Descent in the\ninfinite-dimensional space, gives a faster convergence rate compared to that of\nother random features. In particular, we show that by drawing $R$ random grids\nwith at least $\\kappa$ number of non-empty bins per grid in expectation, RB\nmethod achieves a convergence rate of $O(1/(\\kappa R))$, which not only\nsharpens its $O(1/\\sqrt{R})$ rate from Monte Carlo analysis, but also shows a\n$\\kappa$ times speedup over other random features under the same analysis\nframework. In addition, we demonstrate another advantage of RB in the\nL1-regularized setting, where unlike other random features, a RB-based\nCoordinate Descent solver can be parallelized with guaranteed speedup\nproportional to $\\kappa$. Our extensive experiments demonstrate the superior\nperformance of the RB features over other random features and kernel\napproximation methods. Our code and data is available at {\n\\url{https://github.com/teddylfwu/RB_GEN}}.","url_abs":"http://arxiv.org/abs/1809.05247v2","url_pdf":"http://arxiv.org/pdf/1809.05247v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-random-binning-features-fast","repo_url":"https://github.com/teddylfwu/RB_GEN","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"revisiting-random-binning-features-fast","repo_url":"https://github.com/teddylfwu/RandomBinning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}