Papers › NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension
Xin He, Jiangchao Yao, Yuxin Wang, Zhenheng Tang, Ka Chu Cheung, Simon See, Bo Han, Xiaowen Chu
One-shot neural architecture search (NAS) substantially improves the search efficiency by training one supernet to estimate the performance of every possible child architecture (i.e., subnet). However, the inconsistency of characteristics among subnets incurs serious interference in the optimization, resulting in poor performance ranking correlation of subnets. Subsequent explorations decompose supernet weights via a particular criterion, e.g., gradient matching, to reduce the interference; yet they suffer from huge computational cost and low space separability. In this work, we propose a lightweight and effective local intrinsic dimension (LID)-based method NAS-LID. NAS-LID evaluates the geometrical properties of architectures by calculating the low-cost LID features layer-by-layer, and the similarity characterized by LID enjoys better separability compared with gradients, which thus effectively reduces the interference among subnets. Extensive experiments on NASBench-201 indicate that NAS-LID achieves superior performance with better efficiency. Specifically, compared to the gradient-driven method, NAS-LID can save up to 86% of GPU memory overhead when searching on NASBench-201. We also demonstrate the effectiveness of NAS-LID on ProxylessNAS and OFA spaces. Source code: https://github.com/marsggbo/NAS-LID.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
1 archive task tag without a task page not shown.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | NAS-LID+RSPS | Accuracy (Test) | 92.9 | #26 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-10 | NAS-LID+RSPS | Accuracy (Val) | 89.74 | #26 of 37 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | NAS-LID+RSPS | Accuracy (Test) | 69.39 | #29 of 40 | Archive leaderboard | report |
| Neural Architecture Search | NAS-Bench-201, CIFAR-100 | NAS-LID+RSPS | Accuracy (Val) | 69.38 | #29 of 40 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections