{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gxnor-net-training-deep-neural-networks-with","title":"GXNOR-Net: Training deep neural networks with ternary weights and activations without full-precision memory under a unified discretization framework","arxiv_id":"1705.09283","date":"2017-05-25","proceeding":null,"authors":["Lei Deng","Peng Jiao","Jing Pei","Zhenzhi Wu","Guoqi Li"],"abstract":"There is a pressing need to build an architecture that could subsume these\nnetworks under a unified framework that achieves both higher performance and\nless overhead. To this end, two fundamental issues are yet to be addressed. The\nfirst one is how to implement the back propagation when neuronal activations\nare discrete. The second one is how to remove the full-precision hidden weights\nin the training phase to break the bottlenecks of memory/computation\nconsumption. To address the first issue, we present a multi-step neuronal\nactivation discretization method and a derivative approximation technique that\nenable the implementing the back propagation algorithm on discrete DNNs. While\nfor the second issue, we propose a discrete state transition (DST) methodology\nto constrain the weights in a discrete space without saving the hidden weights.\nThrough this way, we build a unified framework that subsumes the binary or\nternary networks as its special cases, and under which a heuristic algorithm is\nprovided at the website https://github.com/AcrossV/Gated-XNOR. More\nparticularly, we find that when both the weights and activations become ternary\nvalues, the DNNs can be reduced to sparse binary networks, termed as gated XNOR\nnetworks (GXNOR-Nets) since only the event of non-zero weight and non-zero\nactivation enables the control gate to start the XNOR logic operations in the\noriginal binary networks. This promises the event-driven hardware design for\nefficient mobile intelligence. We achieve advanced performance compared with\nstate-of-the-art algorithms. Furthermore, the computational sparsity and the\nnumber of states in the discrete space can be flexibly modified to make it\nsuitable for various hardware platforms.","url_abs":"http://arxiv.org/abs/1705.09283v5","url_pdf":"http://arxiv.org/pdf/1705.09283v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gxnor-net-training-deep-neural-networks-with","repo_url":"https://github.com/AcrossV/Gated-XNOR","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1705.09283","atlas_url":"https://app.syntology.ai/?focus=1705.09283","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}