Two-Phase Iteration for Value Function Approximation and Hyperparameter Optimization in Gaussian-Kernel-Based Adaptive Critic Design

Algorithm 2

Refinement of sample set.

  l = 1         

  Calculate and using (6) and (7)

  Calculate using (40)

Sort all , in ascending order

Add the first state-action pairs to sample set to get extended set

  L + >

  Calculate hyperparameters based on

     l = 1     +

    Calculate and var using (6) and (7)

    Calculate using (40)

  Sort all , in descending order

  Delete the first + − samples to get refined set