{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-a-code-machine-learning-for","title":"Learning a Code: Machine Learning for Approximate Non-Linear Coded Computation","arxiv_id":"1806.01259","date":"2018-06-04","proceeding":null,"authors":["Jack Kosaian","K. V. Rashmi","Shivaram Venkataraman"],"abstract":"Machine learning algorithms are typically run on large scale, distributed\ncompute infrastructure that routinely face a number of unavailabilities such as\nfailures and temporary slowdowns. Adding redundant computations using\ncoding-theoretic tools called \"codes\" is an emerging technique to alleviate the\nadverse effects of such unavailabilities. A code consists of an encoding\nfunction that proactively introduces redundant computation and a decoding\nfunction that reconstructs unavailable outputs using the available ones. Past\nwork focuses on using codes to provide resilience for linear computations and\nspecific iterative optimization algorithms. However, computations performed for\na variety of applications including inference on state-of-the-art machine\nlearning algorithms, such as neural networks, typically fall outside this\nrealm. In this paper, we propose taking a learning-based approach to designing\ncodes that can handle non-linear computations. We present carefully designed\nneural network architectures and a training methodology for learning encoding\nand decoding functions that produce approximate reconstructions of unavailable\ncomputation results. We present extensive experimental results demonstrating\nthe effectiveness of the proposed approach: we show that the our learned codes\ncan accurately reconstruct $64 - 98\\%$ of the unavailable predictions from\nneural-network based image classifiers on the MNIST, Fashion-MNIST, and\nCIFAR-10 datasets. To the best of our knowledge, this work proposes the first\nlearning-based approach for designing codes, and also presents the first\ncoding-theoretic solution that can provide resilience for any non-linear\n(differentiable) computation. Our results show that learning can be an\neffective technique for designing codes, and that learned codes are a highly\npromising approach for bringing the benefits of coding to non-linear\ncomputations.","url_abs":"http://arxiv.org/abs/1806.01259v1","url_pdf":"http://arxiv.org/pdf/1806.01259v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-a-code-machine-learning-for","repo_url":"https://github.com/Thesys-lab/learned-cc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"learning-a-code-machine-learning-for","repo_url":"https://github.com/Thesys-lab/learned-coded-computation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"learning-a-code-machine-learning-for","repo_url":"https://github.com/Thesys-lab/parity-models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.01259","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}