{"url":"/method/mesh-tensorflow","slug":"mesh-tensorflow","name":"Mesh-TensorFlow","full_name":"Mesh-TensorFlow","full_name_withheld":false,"description_markdown":"**Mesh-TensorFlow** is a language for specifying a general class of distributed tensor computations. Where data-parallelism can be viewed as splitting tensors and operations along the \"batch\" dimension, in Mesh-TensorFlow, the user can specify any tensor dimensions to be split across any dimensions of a multi-dimensional mesh of processors. A MeshTensorFlow graph compiles into a SPMD program consisting of parallel operations coupled with collective communication primitives such as Allreduce.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Mesh-TensorFlow: Deep Learning for Supercomputers","paper":"/paper/mesh-tensorflow-deep-learning-for","first_author":"Noam Shazeer","n_authors":12,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/mesh-tensorflow-deep-learning-for"},"source":{"url":"http://arxiv.org/abs/1811.02084v1","title":"Mesh-TensorFlow: Deep Learning for Supercomputers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Intra-Layer Parallel","url":"/methods/category/intra-layer-parallel","pwc_aliases":[]},{"area":"General","area_id":"general","collection":"Model Parallel Methods","url":"/methods/category/model-parallel-methods","pwc_aliases":[]},{"area":"General","area_id":"general","collection":"Distributed Methods","url":"/methods/category/distributed-methods","pwc_aliases":[]}],"n_papers_tagged":2,"archive_num_papers":2,"papers_newest_first":[{"paper":"/paper/high-resolution-medical-image-analysis-with","title":"High Resolution Medical Image Analysis with Spatial Partitioning","date":"2019-09-06","arxiv_id":"1909.03108","n_code_links":1,"syntology":null},{"paper":"/paper/mesh-tensorflow-deep-learning-for","title":"Mesh-TensorFlow: Deep Learning for Supercomputers","date":"2018-11-05","arxiv_id":"1811.02084","n_code_links":1,"syntology":null}],"papers_shown":2,"tasks":[{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":null,"name":"GPU","papers":1},{"task":"/task/language-modeling","name":"Language Modeling","papers":1},{"task":"/task/language-modelling","name":"Language Modelling","papers":1},{"task":"/task/medical-image-analysis","name":"Medical Image Analysis","papers":1},{"task":"/task/high","name":"Vocal Bursts Intensity Prediction","papers":1}],"tasks_shown":6,"n_tasks":6,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/mesh-tensorflow"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}