{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2601-21579","title":"KromHC: Manifold-Constrained Hyper-Connections with Kronecker-Product Residual Matrices","arxiv_id":"2601.21579","date":"2026-01-29","proceeding":"ICML","authors":["Wuyang Zhou","Yuxuan Gu","Giorgos Iacovides","Danilo Mandic"],"abstract":"The success of Hyper-Connections (HC) in neural networks (NN) has also highlighted issues related to training instability and restricted scalability. The Manifold-Constrained Hyper-Connections (mHC) mitigate these challenges by projecting the residual connection space onto a Birkhoff polytope, however, it faces two issues: 1) its iterative Sinkhorn-Knopp (SK) algorithm does not always yield exactly doubly stochastic residual matrices; 2) mHC incurs a prohibitive $O(n^3C)$ parameter complexity with $n$ as the width of the residual stream and $C$ as the feature dimension. The recently proposed mHC-lite reparametrizes the residual matrix via the Birkhoff-von-Neumann theorem to guarantee double stochasticity, but also faces a factorial explosion in its parameter complexity, $O \\left( nC \\cdot n! \\right)$. To address both challenges, we propose KromHC, which uses the Kronecker products of smaller doubly stochastic matrices to parametrize the residual matrix in mHC. By enforcing manifold constraints across the factor residual matrices along each mode of the tensorized residual stream, KromHC guarantees exact double stochasticity of the residual matrices while reducing parameter complexity to only $O(n^2C)$. Experiments show that KromHC matches or even outperforms other state-of-the-art (SOTA) mHC variants, while requiring significantly fewer trainable parameters. The code is at https://github.com/wz1119/KromHC.","url_abs":"https://arxiv.org/abs/2601.21579","url_pdf":"https://arxiv.org/pdf/2601.21579","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2601.21579","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2601.21579"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/wz1119/KromHC","reach":null}],"summary":{"ran":2,"ran_honours":5,"ran_violates":3},"by_repo_kind":{"found_in_text":{"samples":7,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"5fd0231265c06c70","entry":"KromHC","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5fd0231265c06c70"}},{"code_sha256_prefix":"9570166c272034c8","entry":"RMSNorm","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9570166c272034c8"}},{"code_sha256_prefix":"1dacda5180fc1b73","entry":"add","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1dacda5180fc1b73"}},{"code_sha256_prefix":"99b3563ebc6734ad","entry":"default","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"99b3563ebc6734ad"}},{"code_sha256_prefix":"f912c209e9c899ca","entry":"divisible_by","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"f912c209e9c899ca"}},{"code_sha256_prefix":"b7c3487f192e31b7","entry":"exists","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"b7c3487f192e31b7"}},{"code_sha256_prefix":"67933792314b7ce3","entry":"factorize_into_twos","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67933792314b7ce3"}},{"code_sha256_prefix":"aa8a6fda7ca15496","entry":"get_2x2_perm_matrices","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa8a6fda7ca15496"}},{"code_sha256_prefix":"a8d0d52939de92d4","entry":"get_all_permutations","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8d0d52939de92d4"}},{"code_sha256_prefix":"a34a9f4e2d4b6799","entry":"get_cached_2x2_perms","repo":"wz1119/KromHC","repo_kind":"found_in_text","path":"hyper_conn/Kromhc.py","file_url":"https://github.com/wz1119/KromHC/blob/HEAD/hyper_conn/Kromhc.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a34a9f4e2d4b6799"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}