VOVU: A Method for Predicting Generalization in Deep Neural Networks

Figure 2

The dynamic trend of the test loss and VOVU trained on Fashion-MNIST in fully connected networks. The point when VOVU begins to rise is nearly the same point that the test loss starts to rise (the learning rate is set to 0.008 and 0.01 and 0.012 from left to right, respectively; the variance of the initialized weights is fixed at 0.1 in each network). (a) Learning rate = 0.008. (b) Learning rate = 0.01. (c) Learning rate = 0.012.

(a)

(b)

(c)