{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/compressing-deep-neural-networks-by-matrix","title":"Compressing deep neural networks by matrix product operators","arxiv_id":"1904.06194","date":"2019-04-11","proceeding":null,"authors":["Ze-Feng Gao","Song Cheng","Rong-Qiang He","Z. Y. Xie","Hui-Hai Zhao","Zhong-Yi Lu","Tao Xiang"],"abstract":"A deep neural network is a parametrization of a multilayer mapping of signals in terms of many alternatively arranged linear and nonlinear transformations. The linear transformations, which are generally used in the fully connected as well as convolutional layers, contain most of the variational parameters that are trained and stored. Compressing a deep neural network to reduce its number of variational parameters but not its prediction power is an important but challenging problem toward the establishment of an optimized scheme in training efficiently these parameters and in lowering the risk of overfitting. Here we show that this problem can be effectively solved by representing linear transformations with matrix product operators (MPOs), which is a tensor network originally proposed in physics to characterize the short-range entanglement in one-dimensional quantum states. We have tested this approach in five typical neural networks, including FC2, LeNet-5, VGG, ResNet, and DenseNet on two widely used data sets, namely, MNIST and CIFAR-10, and found that this MPO representation indeed sets up a faithful and efficient mapping between input and output signals, which can keep or even improve the prediction accuracy with a dramatically reduced number of parameters. Our method greatly simplifies the representations in deep learning, and opens a possible route toward establishing a framework of modern neural networks which might be simpler and cheaper, but more efficient.","url_abs":"https://arxiv.org/abs/1904.06194v2","url_pdf":"https://arxiv.org/pdf/1904.06194v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"compressing-deep-neural-networks-by-matrix","repo_url":"https://github.com/zfgao66/deeplearning-mpo","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1904.06194","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.06194"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zfgao66/deeplearning-mpo","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fc52a4f8f0232e03","entry":"max_pool_2x2","repo":"zfgao66/deeplearning-mpo","repo_kind":"official","path":"LeNet5/mpo_lenet5/inference.py","file_url":"https://github.com/zfgao66/deeplearning-mpo/blob/HEAD/LeNet5/mpo_lenet5/inference.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fc52a4f8f0232e03"}},{"code_sha256_prefix":"ede253e84b61e886","entry":"batch_normalization","repo":"zfgao66/deeplearning-mpo","repo_kind":"official","path":"DenseNet/densent-mpo/tensornet/layers/batch_normalization.py","file_url":"https://github.com/zfgao66/deeplearning-mpo/blob/HEAD/DenseNet/densent-mpo/tensornet/layers/batch_normalization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ede253e84b61e886"}},{"code_sha256_prefix":"3eb13d07094b10e1","entry":"conv2d","repo":"zfgao66/deeplearning-mpo","repo_kind":"official","path":"LeNet5/mpo_lenet5/inference.py","file_url":"https://github.com/zfgao66/deeplearning-mpo/blob/HEAD/LeNet5/mpo_lenet5/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3eb13d07094b10e1"}},{"code_sha256_prefix":"e461eeb80d497272","entry":"get_var_wrap","repo":"zfgao66/deeplearning-mpo","repo_kind":"official","path":"LeNet5/mpo_lenet5/tt.py","file_url":"https://github.com/zfgao66/deeplearning-mpo/blob/HEAD/LeNet5/mpo_lenet5/tt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e461eeb80d497272"}},{"code_sha256_prefix":"eb3fe697183c4324","entry":"weight_variable","repo":"zfgao66/deeplearning-mpo","repo_kind":"official","path":"LeNet5/mpo_lenet5/inference.py","file_url":"https://github.com/zfgao66/deeplearning-mpo/blob/HEAD/LeNet5/mpo_lenet5/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb3fe697183c4324"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}