An Empirical Investigation of Transfer Effects for Reinforcement Learning

Algorithm 1

The Q-learning based algorithm for the sorting task. RL_Sort.

input: Straining, Qn[Sn, An]
(1)	initialize
(2)	upper_bound = n + 1
(3)	train_steps = 0
(4)	success_rate = 0.75
(5)	Sgoal = [1, 2, ..., n]
(6)	repeat
(7)	end = FALSE
(8)	swap_times = 0
(9)	s = Straining
(10)	current_rate = 0
(11)	repeat
(12)	Select an action a based on ε-greedy
(13)	Perform the action a and observe s′ and the corresponding reward
(14)	swap_times = swap_times + 1
(15)	if (s′ is S_goal) then
(16)	Qn[s, a]  ⟵  Qn [s, a] + α × (reward_win − Qn [s, a])
(17)	end = TRUE
(18)	Check the success rate for the latest 100 episodes and assign to current_rate
(19)	elseif (swap_times >upper_bound) then
(20)
(21)	end = TRUE
(22)	else
(23)	if (dist(s′, S_goal) > dist(s, S_goal))
(24)
(25)	elseif (dist(s′, S_goal) < dist(s, S_goal))
(26)
(27)	else
(28)
(29)	s  ⟵  s′
(30)	until end is TRUE
(31)	train_steps = train_steps + 1
(32)	until current_rate >= success_rate
(33)	return Qn , train_steps