Design and Optimization in MEC-Based Intelligent Rail System by Integration of Distributed Multi-Hop Communication and Blockchain

Algorithm 1

Performance optimization framework for MEC-enabled smart rail system by multi-hop data transmission and blockchain based on dueling DQN.

(1)	Initialization:
	Initialize the experience memory and the mini-batch size ;
	Initialize evaluated network with the weight and bias set ;
	Initialize target network with the weight and bias set ;
	Initialize the greedy coefficient ;
(2)	for do
(3)	Reset the state of trains and MEC servers with a random initial observation , and ;
(4)	fordo
(5)	Randomly choose a probability ;
(6)	ifthen
(7)	Randomly choose an action based on -greedy policy;
(8)	else
(9)	;
(10)	end if
(11)	Execute action and obtain the reward , and proceed to the next observation ;
(12)	Store the experience , , , into experience replay memory;
(13)	Randomly sample a mini-batch of , , , from experience replay memory ;
(14)	Obtain two parts of evaluated network, including and , and merge them as through equation (33);
(15)	Obtain target Q value in target network by ;
(16)	Train evaluated network to minimize loss function by ;
(17)	Every several training steps, modify target Q-network according to evaluated Q-network;
(18)	;
(19)	end for
(20)	end for