Differentiable Network Pruning via Polarization of Probabilistic Channelwise Soft Masks

Figure 1

An illustration of PPSM. The PPSM framework for channel pruning consists of a conditional variational auto-encoder (CVAE), where the encoder learns the posterior distribution of channelwise soft masks given the output features of a baseline network, and the decoder formed by the pruned network learns to recover the baseline features. PPSM combines variational inference with a polarization regularization to effectively learn the posterior distributions of the masks and simultaneously divide the filters into two clearly separated parts, and therefore facilitate the pruning of channels with masks close to zero.