{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/low-rank-lottery-tickets-finding-efficient","title":"Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations","arxiv_id":"2205.13571","date":"2022-05-26","proceeding":null,"authors":["Steffen Schotthöfer","Emanuele Zangrando","Jonas Kusch","Gianluca Ceruti","Francesco Tudisco"],"abstract":"Neural networks have achieved tremendous success in a large variety of applications. However, their memory footprint and computational demand can render them impractical in application settings with limited hardware or energy resources. In this work, we propose a novel algorithm to find efficient low-rank subnetworks. Remarkably, these subnetworks are determined and adapted already during the training phase and the overall time and memory resources required by both training and evaluating them are significantly reduced. The main idea is to restrict the weight matrices to a low-rank manifold and to update the low-rank factors rather than the full matrix during training. To derive training updates that are restricted to the prescribed manifold, we employ techniques from dynamic model order reduction for matrix differential equations. This allows us to provide approximation, stability, and descent guarantees. Moreover, our method automatically and dynamically adapts the ranks during training to achieve the desired approximation accuracy. The efficiency of the proposed method is demonstrated through a variety of numerical experiments on fully-connected and convolutional networks.","url_abs":"https://arxiv.org/abs/2205.13571v2","url_pdf":"https://arxiv.org/pdf/2205.13571v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"low-rank-lottery-tickets-finding-efficient","repo_url":"https://github.com/compile-gssi-lab/dlrt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"low-rank-lottery-tickets-finding-efficient","repo_url":"https://github.com/compilelab/dlrt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"low-rank-lottery-tickets-finding-efficient","repo_url":"https://github.com/csmmlab/dlranet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"low-rank-lottery-tickets-finding-efficient","repo_url":"https://github.com/scsteffen/dlrt-net","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2205.13571","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.13571"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/scsteffen/dlrt-net","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/compile-gssi-lab/dlrt","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/csmmlab/dlranet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/compilelab/dlrt","reach":{"status":"ok"}}],"summary":{"ran_honours":2,"unverified":3},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1},"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"7816c6d65ab55d8f","entry":"get_angles","repo":"scsteffen/dlrt-net","repo_kind":"listed","path":"src/networks/transformer_dlrt.py","file_url":"https://github.com/scsteffen/dlrt-net/blob/HEAD/src/networks/transformer_dlrt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7816c6d65ab55d8f"}},{"code_sha256_prefix":"70fe09020aeba7f4","entry":"positional_encoding","repo":"scsteffen/dlrt-net","repo_kind":"listed","path":"src/networks/transformer_dlrt.py","file_url":"https://github.com/scsteffen/dlrt-net/blob/HEAD/src/networks/transformer_dlrt.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70fe09020aeba7f4"}},{"code_sha256_prefix":"b96fc548499f7aef","entry":"create_padding_mask","repo":"scsteffen/dlrt-net","repo_kind":"listed","path":"src/networks/transformer_dlrt.py","file_url":"https://github.com/scsteffen/dlrt-net/blob/HEAD/src/networks/transformer_dlrt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b96fc548499f7aef"}},{"code_sha256_prefix":"7a4c362b07867a72","entry":"filter_max_tokens","repo":"csmmlab/dlranet","repo_kind":"official","path":"src/speech_transformer_DLRT.py","file_url":"https://github.com/csmmlab/dlranet/blob/HEAD/src/speech_transformer_DLRT.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a4c362b07867a72"}},{"code_sha256_prefix":"0bc1827082ea37b5","entry":"normalize_img","repo":"csmmlab/dlranet","repo_kind":"official","path":"src/mnist_DLRT.py","file_url":"https://github.com/csmmlab/dlranet/blob/HEAD/src/mnist_DLRT.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bc1827082ea37b5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}