{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/intel-ngraph-an-intermediate-representation","title":"Intel nGraph: An Intermediate Representation, Compiler, and Executor for Deep Learning","arxiv_id":"1801.08058","date":"2018-01-24","proceeding":null,"authors":["Scott Cyphers","Arjun K. Bansal","Anahita Bhiwandiwalla","Jayaram Bobba","Matthew Brookhart","Avijit Chakraborty","Will Constable","Christian Convey","Leona Cook","Omar Kanawi","Robert Kimball","Jason Knight","Nikolay Korovaiko","Varun Kumar","Yixing Lao","Christopher R. Lishka","Jaikrishnan Menon","Jennifer Myers","Sandeep Aswath Narayana","Adam Procter","Tristan J. Webb"],"abstract":"The Deep Learning (DL) community sees many novel topologies published each\nyear. Achieving high performance on each new topology remains challenging, as\neach requires some level of manual effort. This issue is compounded by the\nproliferation of frameworks and hardware platforms. The current approach, which\nwe call \"direct optimization\", requires deep changes within each framework to\nimprove the training performance for each hardware backend (CPUs, GPUs, FPGAs,\nASICs) and requires $\\mathcal{O}(fp)$ effort; where $f$ is the number of\nframeworks and $p$ is the number of platforms. While optimized kernels for\ndeep-learning primitives are provided via libraries like Intel Math Kernel\nLibrary for Deep Neural Networks (MKL-DNN), there are several compiler-inspired\nways in which performance can be further optimized. Building on our experience\ncreating neon (a fast deep learning library on GPUs), we developed Intel\nnGraph, a soon to be open-sourced C++ library to simplify the realization of\noptimized deep learning performance across frameworks and hardware platforms.\nInitially-supported frameworks include TensorFlow, MXNet, and Intel neon\nframework. Initial backends are Intel Architecture CPUs (CPU), the Intel(R)\nNervana Neural Network Processor(R) (NNP), and NVIDIA GPUs. Currently supported\ncompiler optimizations include efficient memory management and data layout\nabstraction. In this paper, we describe our overall architecture and its core\ncomponents. In the future, we envision extending nGraph API support to a wider\nrange of frameworks, hardware (including FPGAs and ASICs), and compiler\noptimizations (training versus inference optimizations, multi-node and\nmulti-device scaling via efficient sub-graph partitioning, and HW-specific\ncompounding of operations).","url_abs":"http://arxiv.org/abs/1801.08058v2","url_pdf":"http://arxiv.org/pdf/1801.08058v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"intel-ngraph-an-intermediate-representation","repo_url":"https://github.com/NervanaSystems/ngraph-python","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"management","task_name":"Management"},{"task_slug":"math","task_name":"Math"},{"task_slug":"graph-partitioning","task_name":"graph partitioning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1801.08058","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}