{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-k-means-re-training-and-parameter-1","title":"Deep $k$-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutions","arxiv_id":"1806.09228","date":"2018-06-24","proceeding":null,"authors":["Junru Wu","Yue Wang","Zhen-Yu Wu","Zhangyang Wang","Ashok Veeraraghavan","Yingyan Lin"],"abstract":"The current trend of pushing CNNs deeper with convolutions has created a\npressing demand to achieve higher compression gains on CNNs where convolutions\ndominate the computation and parameter amount (e.g., GoogLeNet, ResNet and Wide\nResNet). Further, the high energy consumption of convolutions limits its\ndeployment on mobile devices. To this end, we proposed a simple yet effective\nscheme for compressing convolutions though applying k-means clustering on the\nweights, compression is achieved through weight-sharing, by only recording $K$\ncluster centers and weight assignment indexes. We then introduced a novel\nspectrally relaxed $k$-means regularization, which tends to make hard\nassignments of convolutional layer weights to $K$ learned cluster centers\nduring re-training. We additionally propose an improved set of metrics to\nestimate energy consumption of CNN hardware implementations, whose estimation\nresults are verified to be consistent with previously proposed energy\nestimation tool extrapolated from actual hardware measurements. We finally\nevaluated Deep $k$-Means across several CNN models in terms of both compression\nratio and energy consumption reduction, observing promising results without\nincurring accuracy loss. The code is available at\nhttps://github.com/Sandbox3aster/Deep-K-Means","url_abs":"http://arxiv.org/abs/1806.09228v1","url_pdf":"http://arxiv.org/pdf/1806.09228v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-k-means-re-training-and-parameter-1","repo_url":"https://github.com/Sandbox3aster/Deep-K-Means","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"auxiliary-classifier","method_name":"Auxiliary Classifier"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"googlenet","method_name":"GoogLeNet"},{"method_slug":"inception-module","method_name":"Inception Module"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"k-means-clustering","method_name":"k-Means Clustering"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1806.09228","atlas_url":"https://app.syntology.ai/?focus=1806.09228","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}