A Novel Low-Bit Quantization Strategy for Compressing Deep Neural Networks

Table 5

Sparsity of ResNet-18 on CIFAR10.

Layers (weight tensors)	Full precision (1 − sparsity) (%)	Our method (1 − sparsity) (%)
Conv1 (64, 3, 3, 3)	100	100
Conv2 (64, 64, 3, 3)	100	85.32
Conv3 (64, 64, 3, 3)	100	86.71
Conv4 (64, 64, 3, 3)	100	85.84
Conv5 (64, 64, 3, 3)	100	85.10
Conv6 (128, 64, 3, 3)	100	86.04
Conv7 (128, 128, 3, 3)	100	83.46
Conv8 (128, 64, 1, 1)	100	86.52
Conv9 (128, 128, 3, 3)	100	82.88
Conv10 (128, 128, 3, 3)	100	80.75
Conv11 (256, 128, 3, 3)	100	77.45
Conv12 (256, 256, 3, 3)	100	70.23
Conv13 (256, 128, 1, 1)	100	77.74
Conv14 (256, 256, 3, 3)	100	59.51
Conv15 (256, 256, 3, 3)	100	42.64
Conv16 (512, 256, 3, 3)	100	22.16
Conv17 (512, 512, 3, 3)	100	10.72
Conv18 (512, 256, 1, 1)	100	41.56
Conv19 (512, 512, 3, 3)	100	5.02
Conv20 (512, 512, 3, 3)	100	3.46
1 − Sparsity	100	23.32
Accuracy	93.74	92.52