Research Article

A Caching-Enabled Permissioned Blockchain Scheme for Industrial Internet of Things Based on Deep Reinforcement Learning

Algorithm 4

Train Caching Agent on Computing Node.
Require: Cache Agent , Main Network Update Frequency , Target Network Update Frequency , Blockchain , State Database
1: Initialize the main network of with
2: Initialize the target network of with
3: Initialize experience buffer
4: Initialize timestep
5: for each new transaction or smart contract query request do
6: ifthen
7:   the keys whose values were affected by
8:  ifthen
9:    the keys whose values were read by
10:  for each key in or do
11:    the block that defines the latest value of in
12:   Update the state of according to Equation (2)
13:   Treat the block body of as a candidate
14:   if is not in cache then
15:    Obtain according to Equation (3)
16:    ifthen
17:     
18:    with probability select
19:     
20:    otherwise
21:     Select random from
22:    ifthen
23:     Delete from and update according to Algorithm 3
24:     Address on DHT for the responsible storage nodes by hashing
25:     Request from a storage node with low latency
26:     Insert into and update according to Algorithm 2
27:    Obtain according to Equation (7)
28:    ifthen
29:     Copy to the target network of
30:    ifthen
31:     Sample a mini-batch from
32:     Minimize Equation (11) and update
33: