{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/blaze-simplified-high-performance-cluster","title":"Blaze: Simplified High Performance Cluster Computing","arxiv_id":"1902.01437","date":"2019-02-04","proceeding":null,"authors":["Junhao Li","Hang Zhang"],"abstract":"MapReduce and its variants have significantly simplified and accelerated the\nprocess of developing parallel programs. However, most MapReduce\nimplementations focus on data-intensive tasks while many real-world tasks are\ncompute intensive and their data can fit distributedly into the memory. For\nthese tasks, the speed of MapReduce programs can be much slower than those\nhand-optimized ones. We present Blaze, a C++ library that makes it easy to\ndevelop high performance parallel programs for such compute intensive tasks. At\nthe core of Blaze is a highly-optimized in-memory MapReduce function, which has\nthree main improvements over conventional MapReduce implementations: eager\nreduction, fast serialization, and special treatment for a small fixed key\nrange. We also offer additional conveniences that make developing parallel\nprograms similar to developing serial programs. These improvements make Blaze\nan easy-to-use cluster computing library that approaches the speed of\nhand-optimized parallel code. We apply Blaze to some common data mining tasks,\nincluding word frequency count, PageRank, k-means, expectation maximization\n(Gaussian mixture model), and k-nearest neighbors. Blaze outperforms Apache\nSpark by more than 10 times on average for these tasks, and the speed of Blaze\nscales almost linearly with the number of nodes. In addition, Blaze uses only\nthe MapReduce function and 3 utility functions in its implementation while\nSpark uses almost 30 different parallel primitives in its official\nimplementation.","url_abs":"http://arxiv.org/abs/1902.01437v2","url_pdf":"http://arxiv.org/pdf/1902.01437v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"blaze-simplified-high-performance-cluster","repo_url":"https://github.com/junhao12131/blaze","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"blaze-simplified-high-performance-cluster","repo_url":"https://github.com/jl2922/fgpl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"high","task_name":"Vocal Bursts Intensity Prediction"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}