{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hidden-fold-networks-random-recurrent","title":"Hidden-Fold Networks: Random Recurrent Residuals Using Sparse Supermasks","arxiv_id":"2111.12330","date":"2021-11-24","proceeding":null,"authors":["Ángel López García-Arias","Masanori Hashimoto","Masato Motomura","Jaehoon Yu"],"abstract":"Deep neural networks (DNNs) are so over-parametrized that recent research has found them to already contain a subnetwork with high accuracy at their randomly initialized state. Finding these subnetworks is a viable alternative training method to weight learning. In parallel, another line of work has hypothesized that deep residual networks (ResNets) are trying to approximate the behaviour of shallow recurrent neural networks (RNNs) and has proposed a way for compressing them into recurrent models. This paper proposes blending these lines of research into a highly compressed yet accurate model: Hidden-Fold Networks (HFNs). By first folding ResNet into a recurrent structure and then searching for an accurate subnetwork hidden within the randomly initialized model, a high-performing yet tiny HFN is obtained without ever updating the weights. As a result, HFN achieves equivalent performance to ResNet50 on CIFAR100 while occupying 38.5x less memory, and similar performance to ResNet34 on ImageNet with a memory size 26.8x smaller. The HFN will become even more attractive by minimizing data transfers while staying accurate when it runs on highly-quantized and randomly-weighted DNN inference accelerators. Code available at https://github.com/Lopez-Angel/hidden-fold-networks","url_abs":"https://arxiv.org/abs/2111.12330v1","url_pdf":"https://arxiv.org/pdf/2111.12330v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hidden-fold-networks-random-recurrent","repo_url":"https://github.com/Lopez-Angel/hidden-fold-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.12330","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.12330"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Lopez-Angel/hidden-fold-networks","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"unverified":6},"by_repo_kind":{"official":{"samples":7,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b0f936d4d6ae3b8c","entry":"accuracy","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"utils/eval_utils.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/utils/eval_utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0f936d4d6ae3b8c"}},{"code_sha256_prefix":"8a42a6951b0c5e57","entry":"accumulate","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"utils/net_utils.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/utils/net_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8a42a6951b0c5e57"}},{"code_sha256_prefix":"177a4abf40724f6e","entry":"arg_to_varname","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"configs/parser.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/configs/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"177a4abf40724f6e"}},{"code_sha256_prefix":"ffc9d7cced6f71fb","entry":"argv_to_vars","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"configs/parser.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/configs/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ffc9d7cced6f71fb"}},{"code_sha256_prefix":"13f92b65f6b8000f","entry":"estimate_params_size","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"utils/profiling.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/utils/profiling.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"13f92b65f6b8000f"}},{"code_sha256_prefix":"717f5458343a0b63","entry":"get_lr","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"utils/net_utils.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/utils/net_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"717f5458343a0b63"}},{"code_sha256_prefix":"a99f170ebd682af5","entry":"trim_preceding_hyphens","repo":"Lopez-Angel/hidden-fold-networks","repo_kind":"official","path":"configs/parser.py","file_url":"https://github.com/Lopez-Angel/hidden-fold-networks/blob/HEAD/configs/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a99f170ebd682af5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}