中文版 | English
CSpace
Dynamical Conventional Neural Network Channel Pruning by Genetic Wavelet Channel Search for Image Classification
Chen, Lin1; Gong, Saijun2; Shi, Xiaoyu1; Shang, Mingsheng1
2021-10-27
摘要Neural network pruning is critical to alleviating the high computational cost of deep neural networks on resource-limited devices. Conventional network pruning methods compress the network based on the hand-crafted rules with a pre-defined pruning ratio (PR), which fails to consider the variety of channels among different layers, thus, resulting in a sub-optimal pruned model. To alleviate this issue, this study proposes a genetic wavelet channel search (GWCS) based pruning framework, where the pruning process is modeled as a multi-stage genetic optimization procedure. Its main ideas are 2-fold: (1) it encodes all the channels of the pertained network and divide them into multiple searching spaces according to the different functional convolutional layers from concrete to abstract. (2) it develops a wavelet channel aggregation based fitness function to explore the most representative and discriminative channels at each layer and prune the network dynamically. In the experiments, the proposed GWCS is evaluated on CIFAR-10, CIFAR-100, and ImageNet datasets with two kinds of popular deep convolutional neural networks (CNNs) (ResNet and VGGNet). The results demonstrate that GNAS outperforms state-of-the-art pruning algorithms in both accuracy and compression rate. Notably, GNAS reduces more than 73.1% FLOPs by pruning ResNet-32 with even 0.79% accuracy improvement on CIFAR-100.

关键词neural network pruning neural architecture search wavelet features neural network compression image classification
DOI10.3389/fncom.2021.760554
发表期刊FRONTIERS IN COMPUTATIONAL NEUROSCIENCE
卷号15页码:11
通讯作者Shi, Xiaoyu(xiaoyushi@cigit.ac.cn)
收录类别SCI
WOS记录号WOS:000717617100001
语种英语