{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/how-transferable-are-features-in-deep-neural","title":"How transferable are features in deep neural networks?","arxiv_id":"1411.1792","date":"2014-11-06","proceeding":"NeurIPS 2014 12","authors":["Jason Yosinski","Jeff Clune","Yoshua Bengio","Hod Lipson"],"abstract":"Many deep neural networks trained on natural images exhibit a curious\nphenomenon in common: on the first layer they learn features similar to Gabor\nfilters and color blobs. Such first-layer features appear not to be specific to\na particular dataset or task, but general in that they are applicable to many\ndatasets and tasks. Features must eventually transition from general to\nspecific by the last layer of the network, but this transition has not been\nstudied extensively. In this paper we experimentally quantify the generality\nversus specificity of neurons in each layer of a deep convolutional neural\nnetwork and report a few surprising results. Transferability is negatively\naffected by two distinct issues: (1) the specialization of higher layer neurons\nto their original task at the expense of performance on the target task, which\nwas expected, and (2) optimization difficulties related to splitting networks\nbetween co-adapted neurons, which was not expected. In an example network\ntrained on ImageNet, we demonstrate that either of these two issues may\ndominate, depending on whether features are transferred from the bottom,\nmiddle, or top of the network. We also document that the transferability of\nfeatures decreases as the distance between the base task and target task\nincreases, but that transferring features even from distant tasks can be better\nthan using random features. A final surprising result is that initializing a\nnetwork with transferred features from almost any number of layers can produce\na boost to generalization that lingers even after fine-tuning to the target\ndataset.","url_abs":"http://arxiv.org/abs/1411.1792v1","url_pdf":"http://arxiv.org/pdf/1411.1792v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"how-transferable-are-features-in-deep-neural","repo_url":"https://github.com/Aniket7/Transfer-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"how-transferable-are-features-in-deep-neural","repo_url":"https://github.com/nmningmei/BOLD5000_autoencoder","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"how-transferable-are-features-in-deep-neural","repo_url":"https://github.com/rajs96/ULMFiT-Twitter-US-Airline-Sentiment","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"specificity","task_name":"Specificity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1411.1792","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1411.1792"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nmningmei/BOLD5000_autoencoder","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rajs96/ULMFiT-Twitter-US-Airline-Sentiment","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Aniket7/Transfer-Learning","reach":{"status":"ok"}}],"summary":{"unverified":2},"by_repo_kind":{"listed":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5ff167944e47d734","entry":"cross_correlation","repo":"nmningmei/BOLD5000_autoencoder","repo_kind":"listed","path":"ICA-AROMA/ICA_AROMA_functions.py","file_url":"https://github.com/nmningmei/BOLD5000_autoencoder/blob/HEAD/ICA-AROMA/ICA_AROMA_functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ff167944e47d734"}},{"code_sha256_prefix":"7309169cbee89180","entry":"feature_time_series","repo":"nmningmei/BOLD5000_autoencoder","repo_kind":"listed","path":"ICA-AROMA/ICA_AROMA_functions.py","file_url":"https://github.com/nmningmei/BOLD5000_autoencoder/blob/HEAD/ICA-AROMA/ICA_AROMA_functions.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7309169cbee89180"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}