[1805.12549] Channel Gating Neural Networks

This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that chann

1 mentions: @asam9891
Date: 2019/11/08 09:50

Referring Tweets

@asam9891 Channel Gating Neural Networks 演算しない領域を動的に決めるタイプのdynamic pruning, NeurIPS2019. ReLU の場合、入力が0 になる領域ができるので直感的に演算を省略できる。動的に演算を省略する領域を探しながら推論する。ASIC なら理論値に近い速度で演算可能 t.co/Ycvjirt0Pj

Related Entries

Read more Neural networks for Graph Data NeurIPS2018読み会@PFN
25 users, 9 mentions 2019/01/26 09:46
Read more GitHub - thunlp/GNNPapers: Must-read papers on graph neural networks (GNN)
1 users, 0 mentions 2019/08/18 08:16
Read more Deep Forest :Deep Neural Networkの代替へ向けて - QiitaQiita
0 users, 0 mentions 2018/04/25 17:22
Read more GitHub - NervanaSystems/nlp-architect: A model library for exploring state-of-the-art deep learning ...
3 users, 2 mentions 2019/11/03 14:20
Read more GitHub - nnzhan/Awesome-Graph-Neural-Networks: Paper Lists for Graph Neural Networks
0 users, 2 mentions 2019/05/21 15:48