{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gossip-training-for-deep-learning","title":"Gossip training for deep learning","arxiv_id":"1611.09726","date":"2016-11-29","proceeding":null,"authors":["Michael Blot","David Picard","Matthieu Cord","Nicolas Thome"],"abstract":"We address the issue of speeding up the training of convolutional networks.\nHere we study a distributed method adapted to stochastic gradient descent\n(SGD). The parallel optimization setup uses several threads, each applying\nindividual gradient descents on a local variable. We propose a new way to share\ninformation between different threads inspired by gossip algorithms and showing\ngood consensus convergence properties. Our method called GoSGD has the\nadvantage to be fully asynchronous and decentralized. We compared our method to\nthe recent EASGD in \\cite{elastic} on CIFAR-10 show encouraging results.","url_abs":"http://arxiv.org/abs/1611.09726v1","url_pdf":"http://arxiv.org/pdf/1611.09726v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gossip-training-for-deep-learning","repo_url":"https://github.com/uoguelph-mlrg/Theano-MPI","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.09726","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}