{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/progressive-stochastic-binarization-of-deep","title":"Progressive Stochastic Binarization of Deep Networks","arxiv_id":"1904.02205","date":"2019-04-03","proceeding":null,"authors":["David Hartmann","Michael Wand"],"abstract":"A plethora of recent research has focused on improving the memory footprint\nand inference speed of deep networks by reducing the complexity of (i)\nnumerical representations (for example, by deterministic or stochastic\nquantization) and (ii) arithmetic operations (for example, by binarization of\nweights).\n  We propose a stochastic binarization scheme for deep networks that allows for\nefficient inference on hardware by restricting itself to additions of small\nintegers and fixed shifts. Unlike previous approaches, the underlying\nrandomized approximation is progressive, thus permitting an adaptive control of\nthe accuracy of each operation at run-time. In a low-precision setting, we\nmatch the accuracy of previous binarized approaches. Our representation is\nunbiased - it approaches continuous computation with increasing sample size. In\na high-precision regime, the computational costs are competitive with previous\nquantization schemes. Progressive stochastic binarization also permits\nlocalized, dynamic accuracy control within a single network, thereby providing\na new tool for adaptively focusing computational attention.\n  We evaluate our method on networks of various architectures, already\npretrained on ImageNet. With representational costs comparable to previous\nschemes, we obtain accuracies close to the original floating point\nimplementation. This includes pruned networks, except the known special case of\ncertain types of separated convolutions. By focusing computational attention\nusing progressive sampling, we reduce inference costs on ImageNet further by a\nfactor of up to 33% (before network pruning).","url_abs":"http://arxiv.org/abs/1904.02205v1","url_pdf":"http://arxiv.org/pdf/1904.02205v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"progressive-stochastic-binarization-of-deep","repo_url":"https://github.com/JGU-VC/progressive_stochastic_binarization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"binarization","task_name":"Binarization"},{"task_slug":"network-pruning","task_name":"Network Pruning"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}