{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-generalizable-device-placement","title":"Learning Generalizable Device Placement Algorithms for Distributed Machine Learning","arxiv_id":null,"date":"2019-12-01","proceeding":"NeurIPS 2019 12","authors":["Ravichandra Addanki","Shaileshh Bojja Venkatakrishnan","Shreyan Gupta","Hongzi Mao","Mohammad Alizadeh"],"abstract":"We present Placeto, a reinforcement learning (RL) approach to\nefficiently find device placements for distributed neural network training.\n\nUnlike prior approaches that only find a device placement for a specific computation graph, Placeto can learn generalizable device placement policies that can be applied to any graph.\n\nWe propose two key ideas in our approach:\n(1) we represent the policy as performing iterative placement improvements, rather than outputting a placement in one shot;\n(2) we use graph embeddings to capture relevant information about the  structure of the computation graph, without relying on node labels for indexing.\n\nThese ideas allow Placeto to train efficiently and generalize\nto unseen graphs.\n\nOur experiments show that Placeto requires up to 6.1x\nfewer training steps to find placements that are on par with or\nbetter than the best placements found by prior approaches.\nMoreover, Placeto is able to learn a generalizable placement policy for any given family of graphs that can be used without any re-training to predict optimized placements for unseen graphs from the same family. \n\nThis eliminates the huge overhead incurred by the prior RL approaches whose lack of generalizability necessitates re-training from scratch every time a new graph is to be placed.","url_abs":"http://papers.nips.cc/paper/8653-learning-generalizable-device-placement-algorithms-for-distributed-machine-learning","url_pdf":"http://papers.nips.cc/paper/8653-learning-generalizable-device-placement-algorithms-for-distributed-machine-learning.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-generalizable-device-placement","repo_url":"https://github.com/aravic/generalizable-device-placement","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}