Papers › Group Fisher Pruning for Practical Network Compression

Group Fisher Pruning for Practical Network Compression

2 Aug 2021arXiv:2108.00708archive 2025-07-28

Liyang Liu, Shilong Zhang, Zhanghui Kuang, Aojun Zhou, Jing-Hao Xue, Xinjiang Wang, Yimin Chen, Wenming Yang, Qingmin Liao, Wayne Zhang

Network compression has been widely studied since it is able to reduce the memory and computation cost during inference. However, previous methods seldom deal with complicated structures like residual connections, group/depth-wise convolution and feature pyramid network, where channels of multiple layers are coupled and need to be pruned simultaneously. In this paper, we present a general channel pruning approach that can be applied to various complicated structures. Particularly, we propose a layer grouping algorithm to find coupled channels automatically. Then we derive a unified metric based on Fisher information to evaluate the importance of a single channel and coupled channels. Moreover, we find that inference speedup on GPUs is more correlated with the reduction of memory rather than FLOPs, and thus we employ the memory reduction of each channel to normalize the importance. Our method can be used to prune any structures including those with coupled channels. We conduct extensive experiments on various backbones, including the classic ResNet and ResNeXt, mobile-friendly MobileNetV2, and the NAS-based RegNet, both on image classification and object detection which is under-explored. Experimental results validate that our method can effectively prune sophisticated networks, boosting inference speed without sacrificing accuracy.

Paper PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

jshilong/FisherPruning officialmentioned in paperpytorch report

Ben-Louis/FisherPruning-Pytorch pytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Image Classification Network Pruning Object Detection image-classification object-detection

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Network Pruning	ImageNet	RegX-1.6G	Accuracy	77.97	#4 of 16	Archive leaderboard	report
Network Pruning	ImageNet	RegX-1.6G	GFLOPs	1.588	#4 of 16	Archive leaderboard	report
Network Pruning	ImageNet	RegX-1.6G	MParams	9.3	#4 of 16	Archive leaderboard	report
Network Pruning	ImageNet	MobileNetV2	Accuracy	73.42	#13 of 16	Archive leaderboard	report
Network Pruning	ImageNet	MobileNetV2	GFLOPs	0.29	#13 of 16	Archive leaderboard	report
Network Pruning	ImageNet	MobileNetV2	MParams	3.31	#13 of 16	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization Bottleneck Residual Block Convolution Depthwise Convolution Depthwise Separable Convolution Global Average Pooling Grouped Convolution Inverted Residual Block Kaiming Initialization Max Pooling Pointwise Convolution Pruning ReLU ResNeXt ResNeXt Block Residual Block Residual Connection

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections