AdaCN: An Adaptive Cubic Newton Method for Nonconvex Stochastic Optimization

Table 5

Test accuracy of 1, 2, 3-layer LSTM on Penn Treebank dataset.

Model	Adam	SGD	AdaBound	Apollo	AdaCN
1-layer LSTM	86.86	86.14	85.68	84.90	83.40
2-layer LSTM	68.81	68.54	68.58	69.71	68.17
3-layer LSTM	64.91	64.50	65.18	66.37	63.26

The best results are shown in bold.