Kernel Recursive Least-Squares Temporal Difference Algorithms with Sparsification and Regularization

Algorithm 3

OSKRLSTD-.

)  Input: to be evaluated, , , , , , , ,

()  for    do

()    if    then

()    Initialize ,

()    Take given by , and observe ,

()    Initialize

()    Initialize

()    Initialize

()    Perform Algorithm 2

()    else

()    Take given by , and observe ,

()    Update ,

()    Update , by (38) and (20)

()    ,

()    Perform Algorithm 2

()    if   satisfies the sparsification condition  then

()      , ,

()      Compute , , and by (23)

()      Update , by (39) and (24)

()      Perform Algorithm 2

()    end if

()    end if

()   

() end for