VOVU: A Method for Predicting Generalization in Deep Neural Networks

Figure 3

The dynamic trend of the test loss and VOVU trained on Fashion-MNIST in fully connected networks.The point when VOVU begins to rise is nearly the same point that the test loss starts to rise (the variance of truncated Gaussian distribution of the initialized weight matrix in each nets is 0.05 and 0.1 and 0.15 from left to right, respectively). (a) Variance = 0.05. (b) Variance = 0.1. (c) Variance = 0.15.

(a)

(b)

(c)