{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-fly-operation-batching-in-dynamic","title":"On-the-fly Operation Batching in Dynamic Computation Graphs","arxiv_id":"1705.07860","date":"2017-05-22","proceeding":"NeurIPS 2017 12","authors":["Graham Neubig","Yoav Goldberg","Chris Dyer"],"abstract":"Dynamic neural network toolkits such as PyTorch, DyNet, and Chainer offer\nmore flexibility for implementing models that cope with data of varying\ndimensions and structure, relative to toolkits that operate on statically\ndeclared computations (e.g., TensorFlow, CNTK, and Theano). However, existing\ntoolkits - both static and dynamic - require that the developer organize the\ncomputations into the batches necessary for exploiting high-performance\nalgorithms and hardware. This batching task is generally difficult, but it\nbecomes a major hurdle as architectures become complex. In this paper, we\npresent an algorithm, and its implementation in the DyNet toolkit, for\nautomatically batching operations. Developers simply write minibatch\ncomputations as aggregations of single instance computations, and the batching\nalgorithm seamlessly executes them, on the fly, using computationally efficient\nbatched operations. On a variety of tasks, we obtain throughput similar to that\nobtained with manual batches, as well as comparable speedups over\nsingle-instance learning on architectures that are impractical to batch\nmanually.","url_abs":"http://arxiv.org/abs/1705.07860v1","url_pdf":"http://arxiv.org/pdf/1705.07860v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-fly-operation-batching-in-dynamic","repo_url":"https://github.com/bplank/bilstm-aux","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"on-the-fly-operation-batching-in-dynamic","repo_url":"https://github.com/sslotin/tinkoff-dl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1705.07860","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}