{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-07378","title":"Zero-Shot Neural Network Evaluation with Sample-Wise Activation Patterns","arxiv_id":"2605.07378","date":"2026-05-08","proceeding":null,"authors":["Yameng Peng","Andy Song","HaythamM. Fayek","Vic Ciesielski","Xiaojun Chang"],"abstract":"Zero-shot proxies, also known as training-free metrics, are widely adopted to reduce the computational overhead in neural network evaluation for scenarios such as Neural Architecture Search (NAS), as they do not require any training. Existing zero-shot metrics have several limitations, including weak correlation with the true performance and poor generalisation across different networks or downstream tasks. For example, most of these metrics apply only to either convolutional neural networks (CNNs) or Transformers, but not both. To address these limitations, we propose Sample-Wise Activation Patterns (SWAP), and its derivative, SWAP-Score, a novel and highly effective zero-shot metric. SWAP-Score is broadly applicable across both architecture families and task domains, demonstrating strong predictive performance in the majority of tasks. This metric measures the expressivity of neural networks over a mini-batch of samples, showing a high correlation with the neural networks' ground-truth performance. For both CNNs and Transformers, the SWAP-Score outperforms existing zero-shot metrics across computer vision and natural language processing tasks. For instance, Spearman's correlation coefficient between the SWAP-Score and CIFAR-10 validation accuracy for DARTS CNNs is 0.93, and 0.71 for FlexiBERT Transformers on GLUE tasks. Moreover, SWAP-Score is label-independent, hence can be applied at the pre-training stage of language models to estimate their performance for downstream tasks. When applied to NAS, SWAP-empowered NAS, SWAP-NAS can achieve competitive performance using only approximately 6 and 9 minutes of GPU time, on CIFAR-10 and ImageNet respectively. Our code is available at: https://github.com/pym1024/SWAP_Universal","url_abs":"https://arxiv.org/abs/2605.07378","url_pdf":"https://arxiv.org/pdf/2605.07378","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.07378","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.07378"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/pym1024/SWAP","reach":null}],"summary":{"unverified":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"cf55a5d5878935f2","entry":"SWAP","repo":"pym1024/SWAP","repo_kind":"found_in_text","path":"src/metrics/swap.py","file_url":"https://github.com/pym1024/SWAP/blob/HEAD/src/metrics/swap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AFL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"cf55a5d5878935f2"}},{"code_sha256_prefix":"efa53b6b95613d4d","entry":"SampleWiseActivationPatterns","repo":"pym1024/SWAP","repo_kind":"found_in_text","path":"src/metrics/swap.py","file_url":"https://github.com/pym1024/SWAP/blob/HEAD/src/metrics/swap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AFL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"efa53b6b95613d4d"}},{"code_sha256_prefix":"124cdad1591b2675","entry":"cal_regular_factor","repo":"pym1024/SWAP","repo_kind":"found_in_text","path":"src/metrics/swap.py","file_url":"https://github.com/pym1024/SWAP/blob/HEAD/src/metrics/swap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AFL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"124cdad1591b2675"}},{"code_sha256_prefix":"ded92534b8902560","entry":"count_parameters","repo":"pym1024/SWAP","repo_kind":"found_in_text","path":"src/metrics/swap.py","file_url":"https://github.com/pym1024/SWAP/blob/HEAD/src/metrics/swap.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"AFL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ded92534b8902560"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}