{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/convolutional-neural-networks-with-low-rank","title":"Convolutional neural networks with low-rank regularization","arxiv_id":"1511.06067","date":"2015-11-19","proceeding":null,"authors":["Cheng Tai","Tong Xiao","Yi Zhang","Xiaogang Wang","Weinan E"],"abstract":"Large CNNs have delivered impressive performance in various computer vision\napplications. But the storage and computation requirements make it problematic\nfor deploying these models on mobile devices. Recently, tensor decompositions\nhave been used for speeding up CNNs. In this paper, we further develop the\ntensor decomposition technique. We propose a new algorithm for computing the\nlow-rank tensor decomposition for removing the redundancy in the convolution\nkernels. The algorithm finds the exact global optimizer of the decomposition\nand is more effective than iterative methods. Based on the decomposition, we\nfurther propose a new method for training low-rank constrained CNNs from\nscratch. Interestingly, while achieving a significant speedup, sometimes the\nlow-rank constrained CNNs delivers significantly better performance than their\nnon-constrained counterparts. On the CIFAR-10 dataset, the proposed low-rank\nNIN model achieves $91.31\\%$ accuracy (without data augmentation), which also\nimproves upon state-of-the-art result. We evaluated the proposed method on\nCIFAR-10 and ILSVRC12 datasets for a variety of modern CNNs, including AlexNet,\nNIN, VGG and GoogleNet with success. For example, the forward time of VGG-16 is\nreduced by half while the performance is still comparable. Empirical success\nsuggests that low-rank tensor decompositions can be a very useful tool for\nspeeding up large CNNs.","url_abs":"http://arxiv.org/abs/1511.06067v3","url_pdf":"http://arxiv.org/pdf/1511.06067v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"convolutional-neural-networks-with-low-rank","repo_url":"https://github.com/chengtaipu/lowrankcnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"convolutional-neural-networks-with-low-rank","repo_url":"https://github.com/Aldiom/nn_benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"tensor-decomposition","task_name":"Tensor Decomposition"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"auxiliary-classifier","method_name":"Auxiliary Classifier"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"googlenet","method_name":"GoogLeNet"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"inception-module","method_name":"Inception Module"},{"method_slug":"local-response-normalization","method_name":"Local Response Normalization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1511.06067","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1511.06067"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chengtaipu/lowrankcnn","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Aldiom/nn_benchmark","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"d1d197afa0f8be54","entry":"load_config","repo":"chengtaipu/lowrankcnn","repo_kind":"official","path":"imagenet/lowrank_approx.py","file_url":"https://github.com/chengtaipu/lowrankcnn/blob/HEAD/imagenet/lowrank_approx.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d1d197afa0f8be54"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}