Online Learning for DNN Training: A Stochastic Block Adaptive Gradient Algorithm

Algorithm 1

SBAG.

	Input:
	Parameter: , and where and . denotes coordinate selection probability at time . Moreover, where and .
	Initially Set: and .
	Output:
(1)	fordo
(2)
(3)	Generating diagonal matrix with probability
(4)
(5)	Generating gradient
(6)
(7)
(8)	and
(9)	Clip
(10)
(11)
(12)	end for
(13)	return