{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-equilibrium-models-are-almost-equivalent","title":"Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixtures","arxiv_id":"2402.02697","date":"2024-02-05","proceeding":null,"authors":["Zenan Ling","Longbo Li","Zhanbo Feng","Yixuan Zhang","Feng Zhou","Robert C. Qiu","Zhenyu Liao"],"abstract":"Deep equilibrium models (DEQs), as a typical implicit neural network, have demonstrated remarkable success on various tasks. There is, however, a lack of theoretical understanding of the connections and differences between implicit DEQs and explicit neural network models. In this paper, leveraging recent advances in random matrix theory (RMT), we perform an in-depth analysis on the eigenspectra of the conjugate kernel (CK) and neural tangent kernel (NTK) matrices for implicit DEQs, when the input data are drawn from a high-dimensional Gaussian mixture. We prove, in this setting, that the spectral behavior of these Implicit-CKs and NTKs depend on the DEQ activation function and initial weight variances, but only via a system of four nonlinear equations. As a direct consequence of this theoretical result, we demonstrate that a shallow explicit network can be carefully designed to produce the same CK or NTK as a given DEQ. Despite derived here for Gaussian mixture data, empirical results show the proposed theory and design principle also apply to popular real-world datasets.","url_abs":"https://arxiv.org/abs/2402.02697v2","url_pdf":"https://arxiv.org/pdf/2402.02697v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-equilibrium-models-are-almost-equivalent","repo_url":"https://github.com/stephenli24/inn_eqvi_enn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"deq","method_name":"DEQ"},{"method_slug":"ntk","method_name":"NTK"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.02697","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.02697"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/StephenLi24/INN_eqvi_ENN","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/stephenli24/inn_eqvi_enn","reach":{"status":"ok"}}],"summary":{"ran":10,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"e3900dbe9313f73c","entry":"ED2","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"Gaussian_Integration.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/Gaussian_Integration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3900dbe9313f73c"}},{"code_sha256_prefix":"c6434aacf0ed5279","entry":"Ef2","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"Gaussian_Integration.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/Gaussian_Integration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c6434aacf0ed5279"}},{"code_sha256_prefix":"d9508cbdaae492d6","entry":"Ef2","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9508cbdaae492d6"}},{"code_sha256_prefix":"984d05255e3c4b00","entry":"G1","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"two_layer_lrelu.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/two_layer_lrelu.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"984d05255e3c4b00"}},{"code_sha256_prefix":"a59e6f245a02eca1","entry":"G2","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"one_layer_tanh.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/one_layer_tanh.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a59e6f245a02eca1"}},{"code_sha256_prefix":"d99f92f681d43177","entry":"G2","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"two_layer_lrelu.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/two_layer_lrelu.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d99f92f681d43177"}},{"code_sha256_prefix":"143ea1c1969862d7","entry":"G4","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"two_layer_lrelu.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/two_layer_lrelu.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"143ea1c1969862d7"}},{"code_sha256_prefix":"c9f4a5df7e3cc996","entry":"anderson","repo":"StephenLi24/INN_eqvi_ENN","repo_kind":"official","path":"models.py","file_url":"https://github.com/StephenLi24/INN_eqvi_ENN/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9f4a5df7e3cc996"}},{"code_sha256_prefix":"0f0973bef7e558ef","entry":"g","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"Gaussian_Integration.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/Gaussian_Integration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f0973bef7e558ef"}},{"code_sha256_prefix":"e97510d9597ca6c0","entry":"g","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e97510d9597ca6c0"}},{"code_sha256_prefix":"5c6dba917d1b5a83","entry":"get_tau","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"utils.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c6dba917d1b5a83"}},{"code_sha256_prefix":"c9fcb0f823e70caf","entry":"G1","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"one_layer_tanh.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/one_layer_tanh.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9fcb0f823e70caf"}},{"code_sha256_prefix":"4989374f0d6a1fbc","entry":"G4","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"one_layer_tanh.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/one_layer_tanh.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4989374f0d6a1fbc"}},{"code_sha256_prefix":"61d75f1d5c488836","entry":"gen_data","repo":"stephenli24/inn_eqvi_enn","repo_kind":"official","path":"data_prepare.py","file_url":"https://github.com/stephenli24/inn_eqvi_enn/blob/HEAD/data_prepare.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"61d75f1d5c488836"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}