{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-diverse-model-selection-for","title":"Scalable Diverse Model Selection for Accessible Transfer Learning","arxiv_id":"2111.06977","date":"2021-11-12","proceeding":"NeurIPS 2021 12","authors":["Daniel Bolya","Rohit Mittapalli","Judy Hoffman"],"abstract":"With the preponderance of pretrained deep learning models available off-the-shelf from model banks today, finding the best weights to fine-tune to your use-case can be a daunting task. Several methods have recently been proposed to find good models for transfer learning, but they either don't scale well to large model banks or don't perform well on the diversity of off-the-shelf models. Ideally the question we want to answer is, \"given some data and a source model, can you quickly predict the model's accuracy after fine-tuning?\" In this paper, we formalize this setting as \"Scalable Diverse Model Selection\" and propose several benchmarks for evaluating on this task. We find that existing model selection and transferability estimation methods perform poorly here and analyze why this is the case. We then introduce simple techniques to improve the performance and speed of these algorithms. Finally, we iterate on existing methods to create PARC, which outperforms all other methods on diverse model selection. We have released the benchmarks and method code in hope to inspire future work in model selection for accessible transfer learning.","url_abs":"https://arxiv.org/abs/2111.06977v2","url_pdf":"https://arxiv.org/pdf/2111.06977v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-diverse-model-selection-for","repo_url":"https://github.com/dbolya/parc","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.06977","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2111.06977"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dbolya/parc","reach":null}],"summary":{"ran":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"f795faf39469b430","entry":"PARC","repo":"dbolya/parc","repo_kind":"official","path":"methods.py","file_url":"https://github.com/dbolya/parc/blob/HEAD/methods.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f795faf39469b430"}},{"code_sha256_prefix":"2b012ee5ffe3a573","entry":"feature_reduce","repo":"dbolya/parc","repo_kind":"official","path":"methods.py","file_url":"https://github.com/dbolya/parc/blob/HEAD/methods.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b012ee5ffe3a573"}},{"code_sha256_prefix":"1961129a71f80b5d","entry":"TransferabilityMethod","repo":"dbolya/parc","repo_kind":"official","path":"methods.py","file_url":"https://github.com/dbolya/parc/blob/HEAD/methods.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1961129a71f80b5d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}