{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fractalnet-ultra-deep-neural-networks-without","title":"FractalNet: Ultra-Deep Neural Networks without Residuals","arxiv_id":"1605.07648","date":"2016-05-24","proceeding":null,"authors":["Gustav Larsson","Michael Maire","Gregory Shakhnarovich"],"abstract":"We introduce a design strategy for neural network macro-architecture based on\nself-similarity. Repeated application of a simple expansion rule generates deep\nnetworks whose structural layouts are precisely truncated fractals. These\nnetworks contain interacting subpaths of different lengths, but do not include\nany pass-through or residual connections; every internal signal is transformed\nby a filter and nonlinearity before being seen by subsequent layers. In\nexperiments, fractal networks match the excellent performance of standard\nresidual networks on both CIFAR and ImageNet classification tasks, thereby\ndemonstrating that residual representations may not be fundamental to the\nsuccess of extremely deep convolutional neural networks. Rather, the key may be\nthe ability to transition, during training, from effectively shallow to deep.\nWe note similarities with student-teacher behavior and develop drop-path, a\nnatural extension of dropout, to regularize co-adaptation of subpaths in\nfractal architectures. Such regularization allows extraction of\nhigh-performance fixed-depth subnetworks. Additionally, fractal networks\nexhibit an anytime property: shallow subnetworks provide a quick answer, while\ndeeper subnetworks, with higher latency, provide a more accurate answer.","url_abs":"http://arxiv.org/abs/1605.07648v4","url_pdf":"http://arxiv.org/pdf/1605.07648v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fractalnet-ultra-deep-neural-networks-without","repo_url":"https://github.com/gustavla/fractalnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"fractalnet-ultra-deep-neural-networks-without","repo_url":"https://github.com/jiye-ML/Classify_FRACTALNET","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"fractalnet-ultra-deep-neural-networks-without","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"fractalnet-ultra-deep-neural-networks-without","repo_url":"https://github.com/snf/keras-fractalnet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"droppath","method_name":"DropPath"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"fractal-block","method_name":"Fractal Block"},{"method_slug":"fractalnet","method_name":"FractalNet"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"randomhorizontalflip","method_name":"Random Horizontal Flip"},{"method_slug":"random-resized-crop","method_name":"Random Resized Crop"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"sgd-with-momentum","method_name":"SGD with Momentum"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"step-decay","method_name":"Step Decay"},{"method_slug":"xavier-initialization","method_name":"Xavier Initialization"}],"datasets_introduced":[],"methods_introduced":[{"slug":"droppath","name":"DropPath","full_name":"DropPath"},{"slug":"fractal-block","name":"Fractal Block","full_name":"Fractal Block"},{"slug":"fractalnet","name":"FractalNet","full_name":"FractalNet"}],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"FractalNet-34","rank_in_archive_order":934,"of":1060,"metrics":{"Top 1 Accuracy":"75.88%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-svhn","task":"Image Classification","dataset":"SVHN","model":"FractalNet","rank_in_archive_order":28,"of":62,"metrics":{"Percentage error":"2.01"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1605.07648","atlas_url":"https://app.syntology.ai/?focus=1605.07648","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1605.07648"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiye-ML/Classify_FRACTALNET","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/snf/keras-fractalnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gustavla/fractalnet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/osmr/imgclsmob","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":4,"unverified":2},"by_repo_kind":{"listed":{"samples":6,"ran":4,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"cbc9a548b26d62c9","entry":"conv","repo":"gustavla/fractalnet","repo_kind":"listed","path":"generation/fractalnet.py","file_url":"https://github.com/gustavla/fractalnet/blob/HEAD/generation/fractalnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"cbc9a548b26d62c9"}},{"code_sha256_prefix":"407676128651cf4f","entry":"paint","repo":"gustavla/fractalnet","repo_kind":"listed","path":"generation/fractalnet.py","file_url":"https://github.com/gustavla/fractalnet/blob/HEAD/generation/fractalnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"407676128651cf4f"}},{"code_sha256_prefix":"b3442826ef00d81b","entry":"pool","repo":"gustavla/fractalnet","repo_kind":"listed","path":"generation/fractalnet.py","file_url":"https://github.com/gustavla/fractalnet/blob/HEAD/generation/fractalnet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"b3442826ef00d81b"}},{"code_sha256_prefix":"ff58e87bb50373d8","entry":"tensor_shape","repo":"jiye-ML/Classify_FRACTALNET","repo_kind":"listed","path":"src/fractal_block.py","file_url":"https://github.com/jiye-ML/Classify_FRACTALNET/blob/HEAD/src/fractal_block.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ff58e87bb50373d8"}},{"code_sha256_prefix":"4de38f4b392c1c66","entry":"tensorflow_categorical","repo":"snf/keras-fractalnet","repo_kind":"listed","path":"src/fractalnet.py","file_url":"https://github.com/snf/keras-fractalnet/blob/HEAD/src/fractalnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4de38f4b392c1c66"}},{"code_sha256_prefix":"c1545d890d27b807","entry":"theano_multinomial","repo":"snf/keras-fractalnet","repo_kind":"listed","path":"src/fractalnet.py","file_url":"https://github.com/snf/keras-fractalnet/blob/HEAD/src/fractalnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1545d890d27b807"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}