{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diracnets-training-very-deep-neural-networks","title":"DiracNets: Training Very Deep Neural Networks Without Skip-Connections","arxiv_id":"1706.00388","date":"2017-06-01","proceeding":null,"authors":["Sergey Zagoruyko","Nikos Komodakis"],"abstract":"Deep neural networks with skip-connections, such as ResNet, show excellent\nperformance in various image classification benchmarks. It is though observed\nthat the initial motivation behind them - training deeper networks - does not\nactually hold true, and the benefits come from increased capacity, rather than\nfrom depth. Motivated by this, and inspired from ResNet, we propose a simple\nDirac weight parameterization, which allows us to train very deep plain\nnetworks without explicit skip-connections, and achieve nearly the same\nperformance. This parameterization has a minor computational cost at training\ntime and no cost at all at inference, as both Dirac parameterization and batch\nnormalization can be folded into convolutional filters, so that network becomes\na simple chain of convolution-ReLU pairs. We are able to match ResNet-1001\naccuracy on CIFAR-10 with 28-layer wider plain DiracNet, and closely match\nResNets on ImageNet. Our parameterization also mostly eliminates the need of\ncareful initialization in residual and non-residual networks. The code and\nmodels for our experiments are available at\nhttps://github.com/szagoruyko/diracnets","url_abs":"http://arxiv.org/abs/1706.00388v2","url_pdf":"http://arxiv.org/pdf/1706.00388v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diracnets-training-very-deep-neural-networks","repo_url":"https://github.com/szagoruyko/diracnets","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"diracnets-training-very-deep-neural-networks","repo_url":"https://github.com/escorciav/roi_pooling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"diracnets-training-very-deep-neural-networks","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1706.00388","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1706.00388"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/escorciav/roi_pooling","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/szagoruyko/diracnets","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/osmr/imgclsmob","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran_honours":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"152b486f18bee7d5","entry":"cast","repo":"szagoruyko/diracnets","repo_kind":"official","path":"diracnet.py","file_url":"https://github.com/szagoruyko/diracnets/blob/HEAD/diracnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"152b486f18bee7d5"}},{"code_sha256_prefix":"eb1bb3521299a71f","entry":"conv_params","repo":"szagoruyko/diracnets","repo_kind":"official","path":"diracnet.py","file_url":"https://github.com/szagoruyko/diracnets/blob/HEAD/diracnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eb1bb3521299a71f"}},{"code_sha256_prefix":"56fc1a732300a9e5","entry":"linear_params","repo":"szagoruyko/diracnets","repo_kind":"official","path":"diracnet.py","file_url":"https://github.com/szagoruyko/diracnets/blob/HEAD/diracnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56fc1a732300a9e5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}