An Improvement of Stochastic Gradient Descent Approach for Mean-Variance Portfolio Optimization Problem

Algorithm 5

AMSGrad algorithm.

	Data: given the initial value , the number of samples , the step size , and the tolerance . Set .
	Step 1: evaluate the augmented objective function from (16).
	Step 2: compute the stochastic gradient from (20).
	Step 3: set the random index .
	Step 4: compute the decaying averages of past and past squared gradients from (22) and (23).
	Step 5: calculate the bias-corrected moment estimate based on (30).
	Step 6: update the vector from (31). If , then stop the iteration. Otherwise, set and repeat from Step 1.
	Remark:
	The default values for the decay rates are  = 0.9 and  = 0.999, and the smoothing term is , while the tolerance is , and the learning rate is  = 0.001.