{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tf-replicator-distributed-machine-learning","title":"TF-Replicator: Distributed Machine Learning for Researchers","arxiv_id":"1902.00465","date":"2019-02-01","proceeding":null,"authors":["Peter Buchlovsky","David Budden","Dominik Grewe","Chris Jones","John Aslanides","Frederic Besse","Andy Brock","Aidan Clark","Sergio Gómez Colmenarejo","Aedan Pope","Fabio Viola","Dan Belov"],"abstract":"We describe TF-Replicator, a framework for distributed machine learning\ndesigned for DeepMind researchers and implemented as an abstraction over\nTensorFlow. TF-Replicator simplifies writing data-parallel and model-parallel\nresearch code. The same models can be effortlessly deployed to different\ncluster architectures (i.e. one or many machines containing CPUs, GPUs or TPU\naccelerators) using synchronous or asynchronous training regimes. To\ndemonstrate the generality and scalability of TF-Replicator, we implement and\nbenchmark three very different models: (1) A ResNet-50 for ImageNet\nclassification, (2) a SN-GAN for class-conditional ImageNet image generation,\nand (3) a D4PG reinforcement learning agent for continuous control. Our results\nshow strong scalability performance without demanding any distributed systems\nexpertise of the user. The TF-Replicator programming model will be open-sourced\nas part of TensorFlow 2.0 (see\nhttps://github.com/tensorflow/community/pull/25).","url_abs":"http://arxiv.org/abs/1902.00465v1","url_pdf":"http://arxiv.org/pdf/1902.00465v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tf-replicator-distributed-machine-learning","repo_url":"https://github.com/tensorflow/community","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"continuous-control","task_name":"Continuous Control"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"continuous-control","task_name":"continuous-control"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"d4pg","method_name":"D4PG"},{"method_slug":"n-step-returns","method_name":"N-step Returns"},{"method_slug":"prioritized-experience-replay","method_name":"Prioritized Experience Replay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.00465","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1902.00465"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/community","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"50ac80dd2372bb51","entry":"cygpath","repo":"tensorflow/community","repo_kind":"official","path":"rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","file_url":"https://github.com/tensorflow/community/blob/HEAD/rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"50ac80dd2372bb51"}},{"code_sha256_prefix":"e07e303581bd4892","entry":"get_input","repo":"tensorflow/community","repo_kind":"official","path":"rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","file_url":"https://github.com/tensorflow/community/blob/HEAD/rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e07e303581bd4892"}},{"code_sha256_prefix":"fbac3c23d466ca9b","entry":"run_shell","repo":"tensorflow/community","repo_kind":"official","path":"rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","file_url":"https://github.com/tensorflow/community/blob/HEAD/rfcs/20200624-pluggable-device-for-tensorflow/sample/configure.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fbac3c23d466ca9b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}