{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/salus-fine-grained-gpu-sharing-primitives-for","title":"Salus: Fine-Grained GPU Sharing Primitives for Deep Learning Applications","arxiv_id":"1902.04610","date":"2019-02-12","proceeding":null,"authors":["Peifeng Yu","Mosharaf Chowdhury"],"abstract":"GPU computing is becoming increasingly more popular with the proliferation of\ndeep learning (DL) applications. However, unlike traditional resources such as\nCPU or the network, modern GPUs do not natively support fine-grained sharing\nprimitives. Consequently, implementing common policies such as time sharing and\npreemption are expensive. Worse, when a DL application cannot completely use a\nGPU's resources, the GPU cannot be efficiently shared between multiple\napplications, leading to GPU underutilization.\n  We present Salus to enable two GPU sharing primitives: fast job switching and\nmemory sharing, in order to achieve fine-grained GPU sharing among multiple DL\napplications. Salus implements an efficient, consolidated execution service\nthat exposes the GPU to different DL applications, and enforces fine-grained\nsharing by performing iteration scheduling and addressing associated memory\nmanagement issues. We show that these primitives can then be used to implement\nflexible sharing policies such as fairness, prioritization, and packing for\nvarious use cases. Our integration of Salus with TensorFlow and evaluation on\npopular DL jobs show that Salus can improve the average completion time of DL\ntraining jobs by $3.19\\times$, GPU utilization for hyper-parameter tuning by\n$2.38\\times$, and GPU utilization of DL inference applications by $42\\times$\nover not sharing the GPU and $7\\times$ over NVIDIA MPS with small overhead.","url_abs":"http://arxiv.org/abs/1902.04610v1","url_pdf":"http://arxiv.org/pdf/1902.04610v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"salus-fine-grained-gpu-sharing-primitives-for","repo_url":"https://github.com/SymbioticLab/Salus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"management","task_name":"Management"},{"task_slug":"scheduling","task_name":"Scheduling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}