{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/skew-orthogonal-convolutions","title":"Skew Orthogonal Convolutions","arxiv_id":"2105.11417","date":"2021-05-24","proceeding":null,"authors":["Sahil Singla","Soheil Feizi"],"abstract":"Training convolutional neural networks with a Lipschitz constraint under the $l_{2}$ norm is useful for provable adversarial robustness, interpretable gradients, stable training, etc. While 1-Lipschitz networks can be designed by imposing a 1-Lipschitz constraint on each layer, training such networks requires each layer to be gradient norm preserving (GNP) to prevent gradients from vanishing. However, existing GNP convolutions suffer from slow training, lead to significant reduction in accuracy and provide no guarantees on their approximations. In this work, we propose a GNP convolution layer called Skew Orthogonal Convolution (SOC) that uses the following mathematical property: when a matrix is {\\it Skew-Symmetric}, its exponential function is an {\\it orthogonal} matrix. To use this property, we first construct a convolution filter whose Jacobian is Skew-Symmetric. Then, we use the Taylor series expansion of the Jacobian exponential to construct the SOC layer that is orthogonal. To efficiently implement SOC, we keep a finite number of terms from the Taylor series and provide a provable guarantee on the approximation error. Our experiments on CIFAR-10 and CIFAR-100 show that SOC allows us to train provably Lipschitz, large convolutional neural networks significantly faster than prior works while achieving significant improvements for both standard and certified robust accuracies.","url_abs":"https://arxiv.org/abs/2105.11417v2","url_pdf":"https://arxiv.org/pdf/2105.11417v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"skew-orthogonal-convolutions","repo_url":"https://github.com/singlasahil14/SOC","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.11417","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.11417"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/singlasahil14/SOC","reach":null}],"summary":{"ran_fixture":1,"ran":1,"ran_draft_wrong":6,"unverified":1},"by_repo_kind":{"official":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"304a7d6c629c144b","entry":"fantastic_four","repo":"singlasahil14/soc","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/soc/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"304a7d6c629c144b"}},{"code_sha256_prefix":"5aa5264887e73887","entry":"fantastic_four","repo":"singlasahil14/SOC","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/SOC/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5aa5264887e73887"}},{"code_sha256_prefix":"4551263d3cc7ab39","entry":"init_model","repo":"singlasahil14/soc","repo_kind":"official","path":"train_standard.py","file_url":"https://github.com/singlasahil14/soc/blob/HEAD/train_standard.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4551263d3cc7ab39"}},{"code_sha256_prefix":"d00da20762480770","entry":"l2_normalize","repo":"singlasahil14/soc","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/soc/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d00da20762480770"}},{"code_sha256_prefix":"5f294d9fde22ad26","entry":"l2_normalize","repo":"singlasahil14/SOC","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/SOC/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f294d9fde22ad26"}},{"code_sha256_prefix":"08afa61c0bf4caf7","entry":"robust_statistics","repo":"singlasahil14/soc","repo_kind":"official","path":"train_robust.py","file_url":"https://github.com/singlasahil14/soc/blob/HEAD/train_robust.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"08afa61c0bf4caf7"}},{"code_sha256_prefix":"8472855416e57479","entry":"transpose_filter","repo":"singlasahil14/soc","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/soc/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8472855416e57479"}},{"code_sha256_prefix":"fdef065e9e837d95","entry":"transpose_filter","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fdef065e9e837d95"}},{"code_sha256_prefix":"d547d69d777b49b2","entry":"SOC","repo":"singlasahil14/SOC","repo_kind":"official","path":"skew_ortho_conv.py","file_url":"https://github.com/singlasahil14/SOC/blob/HEAD/skew_ortho_conv.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d547d69d777b49b2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}