{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimizing-deep-learning-recommender-systems","title":"Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures","arxiv_id":"2005.04680","date":"2020-05-10","proceeding":null,"authors":["Dhiraj Kalamkar","Evangelos Georganas","Sudarshan Srinivasan","Jianping Chen","Mikhail Shiryaev","Alexander Heinecke"],"abstract":"During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks. Often this discussion is held in the context of how fast ResNet50 can be trained. Unfortunately, ResNet50 is no longer a representative workload in 2020. Thus, we focus on Recommender Systems which account for most of the AI cycles in cloud computing centers. More specifically, we focus on Facebook's DLRM benchmark. By enabling it to run on latest CPU hardware and software tailored for HPC, we are able to achieve more than two-orders of magnitude improvement in performance (110x) on a single socket compared to the reference CPU implementation, and high scaling efficiency up to 64 sockets, while fitting ultra-large datasets. This paper discusses the optimization techniques for the various operators in DLRM and which component of the systems are stressed by these different operators. The presented techniques are applicable to a broader set of DL workloads that pose the same scaling challenges/characteristics as DLRM.","url_abs":"https://arxiv.org/abs/2005.04680v1","url_pdf":"https://arxiv.org/pdf/2005.04680v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimizing-deep-learning-recommender-systems","repo_url":"https://github.com/hfp/libxsmm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"optimizing-deep-learning-recommender-systems","repo_url":"https://github.com/libxsmm/libxsmm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"cloud-computing","task_name":"Cloud Computing"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}