Trust in Intrusion Detection Systems: An Investigation of Performance Analysis for Machine Learning and Deep Learning Models

Table 17

The performance of deep learning for KDD dataset using LSTM and GRU with selected features.

Model	Evaluation metric	Cross-validation performance					Testing performance
		DOS	Normal	Probe	R2L	U2R	DOS	Normal	Probe	R2L	U2R
LSTM with one layer	TNR	66.96	98.75	100	99.99	100	78.06	98.50	100.00	99.99	100.00
	FPR	33.04	1.25	0	0.01	0	21.94	1.50	0.00	0.01	0.00
	FNR	0.95	30.63	100	79.18	100	1.06	19.57	100.00	77.13	100.00
	Accuracy	92.98	93.51	99.13	99.79	99.99	94.98	95.27	99.15	99.79	99.99
	Precision	92.91	92.35	0	84.04	0.0	95.06	92.09	0.0	82.11	0.0
	Recall	99.05	69.37	0	20.82	0	98.94	80.43	0.00	22.87	0.00
	F-score	95.84	77.59	0.0	39.48	0.0	96.96	85.87	0.0	35.78	0.0
LSTM with two layer	TNR	81.53	98.37	99.99	100	100	85.83	98.37	100.00	99.99	100.00
	FPR	18.47	1.63	0.01	0	0	14.17	1.63	0.00	0.01	0.00
	FNR	1.24	15.93	87.69	94.94	100	1.40	11.50	76.09	76.83	100.00
	Accuracy	95.49	95.82	99.22	99.76	99.99	96.17	96.60	99.35	99.79	99.99
	Precision	95.82	91.78	84.58	91.37	0.0	96.74	92.19	98.18	86.81	0.0
	Recall	98.76	84.07	12.31	5.06	0	98.60	88.50	23.91	23.17	0.00
	F-score	97.26	87.71	33.94	39.52	0.0	97.66	90.31	38.46	36.57	0.0
GRU with one layer	TNR	78.73	99.22	100	100	100	79.00	99.19	100.00	100.00	100.00
	FPR	21.27	0.78	0	0	0	21.00	0.81	0.00	0.00	0.00
	FN	0.36	18.26	100	100	100	0.36	18.15	100.00	100.00	100.00
	Accuracy	95.68	96.11	99.13	99.75	99.99	95.72	96.09	99.15	99.74	99.99
	Precision	95.25	95.77	0.0	0.0	0.0	95.29	95.66	0.0	0.0	0.0
	Recall	99.64	81.74	0	0	0	99.64	81.85	0.00	0.00	0.00
	F-score	97.4	88.2	0.0	0.0	0.0	97.42	88.22	0.0	0.0	0.0
GRU with two layer	TNR	82.9	98.94	99.99	99.98	100	83.25	99.34	99.98	99.99	100.00
	FPR	17.1	1.06	0.01	0.02	0	16.75	0.66	0.02	0.01	0.00
	FNR	0.77	14.4	86.83	73.34	100	0.46	14.11	76.09	76.54	100.00
	Accuracy	96.14	96.56	99.24	99.8	99.99	96.45	96.93	99.34	99.79	99.99
	Precision	96.13	94.62	91.67	80.4	0.0	96.20	96.59	93.08	83.33	0.0
	Recall	99.23	85.6	13.17	26.66	0	99.54	85.89	23.91	23.46	0.00
	F-score	97.65	89.88	35.28	39.92	0.0	97.84	90.92	38.05	36.61	0.0