Papers › Network Pruning via Transformable Architecture Search

Network Pruning via Transformable Architecture Search

23 May 2019NeurIPS 2019 12arXiv:1905.09717archive 2025-07-28

Xuanyi Dong, Yi Yang

Network pruning reduces the computation costs of an over-parameterized network without performance damage. Prevailing pruning algorithms pre-define the width and depth of the pruned networks, and then transfer parameters from the unpruned network to pruned networks. To break the structure limitation of the pruned networks, we propose to apply neural architecture search to search directly for a network with flexible channel and layer sizes. The number of the channels/layers is learned by minimizing the loss of the pruned networks. The feature map of the pruned network is an aggregation of K feature map fragments (generated by K networks of different sizes), which are sampled based on the probability distribution.The loss can be back-propagated not only to the network weights, but also to the parameterized distribution to explicitly tune the size of the channels/layers. Specifically, we apply channel-wise interpolation to keep the feature map with different channel sizes aligned in the aggregation procedure. The maximum probability for the size in each distribution serves as the width and depth of the pruned network, whose parameters are learned by knowledge transfer, e.g., knowledge distillation, from the original networks. Experiments on CIFAR-10, CIFAR-100 and ImageNet demonstrate the effectiveness of our new perspective of network pruning compared to traditional network pruning algorithms. Various searching and knowledge transfer approaches are conducted to show the effectiveness of the two components. Code is at: https://github.com/D-X-Y/NAS-Projects.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

D-X-Y/NAS-Projects officialmentioned in papermentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search (arXiv:2001.00326; that paper's own run record: 15 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 2 violated, 9 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report
D-X-Y/AutoDL-Projects mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets (arXiv:2107.00860; that paper's own run record: 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified); nothing checks that they implement this paper's method. report
D-X-Y/GDAS mentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to One-Shot Neural Architecture Search via Self-Evaluated Template Network (arXiv:1910.05733; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
xxlya/COS598D_Assignment1 mentioned on GitHubpytorchSyntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Knowledge DistillationNetwork PruningNeural Architecture SearchTransfer Learning

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Network Pruning CIFAR-10 TAS-pruned ResNet-110 Accuracy 94.33 #1 of 4 Archive leaderboard report
Network Pruning CIFAR-10 TAS-pruned ResNet-110 GFLOPs 0.119 #1 of 4 Archive leaderboard report
Network Pruning CIFAR-100 TAS-pruned ResNet-110 Accuracy 73.16 #4 of 5 Archive leaderboard report
Network Pruning CIFAR-100 TAS-pruned ResNet-110 GFLOPs 0.12 #4 of 5 Archive leaderboard report
Network Pruning ImageNet TAS-pruned ResNet-50 Accuracy 76.20 #9 of 16 Archive leaderboard report
Network Pruning ImageNet TAS-pruned ResNet-50 GFLOPs 2.3 #9 of 16 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAverage PoolingBatch NormalizationBottleneck Residual BlockConvolutionGlobal Average PoolingKaiming InitializationLSTMMax PoolingPruningReLUResidual BlockResidual ConnectionSigmoid ActivationSoftmaxTanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections