{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/linear-mode-connectivity-and-the-lottery","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","arxiv_id":"1912.05671","date":"2019-12-11","proceeding":"ICML 2020 1","authors":["Jonathan Frankle","Gintare Karolina Dziugaite","Daniel M. Roy","Michael Carbin"],"abstract":"We study whether a neural network optimizes to the same, linearly connected minimum under different samples of SGD noise (e.g., random data order and augmentation). We find that standard vision models become stable to SGD noise in this way early in training. From then on, the outcome of optimization is determined to a linearly connected region. We use this technique to study iterative magnitude pruning (IMP), the procedure used by work on the lottery ticket hypothesis to identify subnetworks that could have trained in isolation to full accuracy. We find that these subnetworks only reach full accuracy when they are stable to SGD noise, which either occurs at initialization for small-scale settings (MNIST) or early in training for large-scale settings (ResNet-50 and Inception-v3 on ImageNet).","url_abs":"https://arxiv.org/abs/1912.05671v4","url_pdf":"https://arxiv.org/pdf/1912.05671v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"linear-mode-connectivity-and-the-lottery","repo_url":"https://github.com/facebookresearch/open_lth","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"linear-mode-connectivity-and-the-lottery","repo_url":"https://github.com/luuyin/lottery-pools","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"linear-mode-connectivity","task_name":"Linear Mode Connectivity"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"auxiliary-classifier","method_name":"Auxiliary Classifier"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"inception-v3","method_name":"Inception-v3"},{"method_slug":"inception-v3-module","method_name":"Inception-v3 Module"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"sgd","method_name":"SGD"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1912.05671","atlas_url":"https://app.syntology.ai/?focus=1912.05671","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}