{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/speeding-up-convolutional-neural-networks","title":"Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition","arxiv_id":"1412.6553","date":"2014-12-19","proceeding":null,"authors":["Vadim Lebedev","Yaroslav Ganin","Maksim Rakhuba","Ivan Oseledets","Victor Lempitsky"],"abstract":"We propose a simple two-step approach for speeding up convolution layers\nwithin large convolutional neural networks based on tensor decomposition and\ndiscriminative fine-tuning. Given a layer, we use non-linear least squares to\ncompute a low-rank CP-decomposition of the 4D convolution kernel tensor into a\nsum of a small number of rank-one tensors. At the second step, this\ndecomposition is used to replace the original convolutional layer with a\nsequence of four convolutional layers with small kernels. After such\nreplacement, the entire network is fine-tuned on the training data using\nstandard backpropagation process.\n  We evaluate this approach on two CNNs and show that it is competitive with\nprevious approaches, leading to higher obtained CPU speedups at the cost of\nlower accuracy drops for the smaller of the two networks. Thus, for the\n36-class character classification CNN, our approach obtains a 8.5x CPU speedup\nof the whole network with only minor accuracy drop (1% from 91% to 90%). For\nthe standard ImageNet architecture (AlexNet), the approach speeds up the second\nconvolution layer by a factor of 4x at the cost of $1\\%$ increase of the\noverall top-5 classification error.","url_abs":"http://arxiv.org/abs/1412.6553v3","url_pdf":"http://arxiv.org/pdf/1412.6553v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/Graphiiz/low-rank-factorization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/Gyiming/MobileSLAM","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/Oliver-jiang/Tensor-decomposition-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/ddfabbro/CNN_CPD","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/ddfabbro/caffe-cp-decomposition","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/ddfabbro/cp-decomposition","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/jacobgil/pytorch-tensor-decompositions","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/keithyuck/Object-Tracking","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/mostafaelhoushi/tensor-decompositions","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"speeding-up-convolutional-neural-networks","repo_url":"https://github.com/timgaripov/TensorNet-TF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":null,"task_name":"CPU"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"tensor-decomposition","task_name":"Tensor Decomposition"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1412.6553","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1412.6553"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ddfabbro/caffe-cp-decomposition","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/timgaripov/TensorNet-TF","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mostafaelhoushi/tensor-decompositions","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ddfabbro/CNN_CPD","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/keithyuck/Object-Tracking","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Graphiiz/low-rank-factorization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Oliver-jiang/Tensor-decomposition-pytorch","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ddfabbro/cp-decomposition","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Gyiming/MobileSLAM","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jacobgil/pytorch-tensor-decompositions","reach":{"status":"ok"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"listed":{"samples":8,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"3fb5447d5e3c1526","entry":"fine_tune","repo":"Graphiiz/low-rank-factorization","repo_kind":"listed","path":"pretrained_main.py","file_url":"https://github.com/Graphiiz/low-rank-factorization/blob/HEAD/pretrained_main.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3fb5447d5e3c1526"}},{"code_sha256_prefix":"ddb40edf84db1df7","entry":"get_per_layer_config","repo":"mostafaelhoushi/tensor-decompositions","repo_kind":"listed","path":"decompositions.py","file_url":"https://github.com/mostafaelhoushi/tensor-decompositions/blob/HEAD/decompositions.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ddb40edf84db1df7"}},{"code_sha256_prefix":"9b8289076669fe4f","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"9b8289076669fe4f"}},{"code_sha256_prefix":"e043983f0912c3b4","entry":"measure_time","repo":"Graphiiz/low-rank-factorization","repo_kind":"listed","path":"pretrained_main.py","file_url":"https://github.com/Graphiiz/low-rank-factorization/blob/HEAD/pretrained_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e043983f0912c3b4"}},{"code_sha256_prefix":"634c4d5814f593b1","entry":"test","repo":"Graphiiz/low-rank-factorization","repo_kind":"listed","path":"pretrained_main.py","file_url":"https://github.com/Graphiiz/low-rank-factorization/blob/HEAD/pretrained_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"634c4d5814f593b1"}},{"code_sha256_prefix":"e8d9d8e78c7f0611","entry":"train","repo":"mostafaelhoushi/tensor-decompositions","repo_kind":"listed","path":"imagenet.py","file_url":"https://github.com/mostafaelhoushi/tensor-decompositions/blob/HEAD/imagenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e8d9d8e78c7f0611"}},{"code_sha256_prefix":"75e61dc3235c8bcc","entry":"train","repo":"mostafaelhoushi/tensor-decompositions","repo_kind":"listed","path":"cifar10.py","file_url":"https://github.com/mostafaelhoushi/tensor-decompositions/blob/HEAD/cifar10.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75e61dc3235c8bcc"}},{"code_sha256_prefix":"242411f7cb8af73b","entry":"validate","repo":"mostafaelhoushi/tensor-decompositions","repo_kind":"listed","path":"imagenet.py","file_url":"https://github.com/mostafaelhoushi/tensor-decompositions/blob/HEAD/imagenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"242411f7cb8af73b"}},{"code_sha256_prefix":"6536aad716b69521","entry":"validate","repo":"mostafaelhoushi/tensor-decompositions","repo_kind":"listed","path":"cifar10.py","file_url":"https://github.com/mostafaelhoushi/tensor-decompositions/blob/HEAD/cifar10.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6536aad716b69521"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}