{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/relaxed-quantization-for-discretized-neural","title":"Relaxed Quantization for Discretized Neural Networks","arxiv_id":"1810.01875","date":"2018-10-03","proceeding":"ICLR 2019 5","authors":["Christos Louizos","Matthias Reisser","Tijmen Blankevoort","Efstratios Gavves","Max Welling"],"abstract":"Neural network quantization has become an important research area due to its\ngreat impact on deployment of large models on resource constrained devices. In\norder to train networks that can be effectively discretized without loss of\nperformance, we introduce a differentiable quantization procedure.\nDifferentiability can be achieved by transforming continuous distributions over\nthe weights and activations of the network to categorical distributions over\nthe quantization grid. These are subsequently relaxed to continuous surrogates\nthat can allow for efficient gradient-based optimization. We further show that\nstochastic rounding can be seen as a special case of the proposed approach and\nthat under this formulation the quantization grid itself can also be optimized\nwith gradient descent. We experimentally validate the performance of our method\non MNIST, CIFAR 10 and Imagenet classification.","url_abs":"http://arxiv.org/abs/1810.01875v1","url_pdf":"http://arxiv.org/pdf/1810.01875v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"relaxed-quantization-for-discretized-neural","repo_url":"https://github.com/newwhitecheng/compress-all-nn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1810.01875","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}