Exploration for Countering the Episodic Memory

Table 1

Median across games of human-normalized scores for listed algorithms at different training frames.

Frames (M)	Nature DQN (%)	Retrace(λ) (%)	A3C (%)	MFEC (%)	CounterEM (MFEC) (%)	NEC (%)	CounterEM (NEC) (%)
1	−0.7	−0.4	0.4	12.8	20.1	16.7	29.6
2	0.0	0.2	0.9	16.7	29.1	27.8	37.6
4	2.4	3.3	1.9	26.6	36.2	36.0	48.4
10	15.7	17.3	3.6	45.4	52.5	54.6	69.3
20	26.8	30.4	7.9	55.9	66.1	72.0	77.6
40	36.7	60.5	18.4	61.9	70.0	83.3	84.5