Trust in Intrusion Detection Systems: An Investigation of Performance Analysis for Machine Learning and Deep Learning Models

Table 16

The performance of deep learning for KDD dataset using LSTM and GRU with all features.

Model	Evaluation metric	Cross-validation performance					Testing performance
		DOS	Normal	Probe	R2L	U2R	DOS	Normal	Probe	R2L	U2R
LSTM with one layer	TNR	98.7	99.87	99.24	91.83	99.88	99.98	99.96	100.00	99.99	100.00
	FPR	1.3	0.13	0.76	8.17	0.12	0.02	0.04	0.00	0.01	0.00
	FNR	10.42	9.64	46.16	0.68	11.67	0.01	0.06	1.24	7.62	66.67
	Accuracy	98.46	99.79	97.47	98.63	99.68	99.99	99.95	99.99	99.97	99.99
	Precision	65.61	86.16	74.48	99.17	93.07	100.00	99.80	99.55	97.22	66.67
	Recall	89.58	90.36	53.84	99.32	88.33	99.99	99.94	98.76	92.38	33.33
	F-score	75.6	88.2	62.28	99.24	90.62	100.00	99.87	99.15	94.74	44.44
LSTM with two layer	TNR	99.89	99.75	100	99.95	100	99.92	99.93	99.99	99.96	100.00
	FPR	0.11	0.25	0	0.05	0	0.08	0.07	0.01	0.04	0.00
	FNR	0.18	0.35	2.66	15.57	100	0.01	0.29	2.22	10.85	100.00
	Accuracy	99.83	99.73	99.97	99.91	99.99	99.98	99.89	99.98	99.93	99.99
	Precision	99.97	98.85	99.58	82.2	0	99.98	99.70	99.37	84.92	0
	Recall	99.82	99.65	97.34	84.43	0	99.99	99.71	97.78	89.15	0.00
	F-score	99.9	99.25	98.44	83.27	0	99.99	99.70	98.57	86.98	0
GRU with one layer	TNR	99.88	99.56	100	99.96	100	99.92	99.90	100.00	99.97	100.00
	FPR	0.12	0.44	0	0.04	0	0.08	0.10	0.00	0.03	0.00
	FNR	0.37	0.32	3.07	14.28	100	0.04	0.25	2.93	11.73	100.00
	Accuracy	99.67	99.58	99.97	99.92	99.99	99.95	99.87	99.97	99.93	99.99
	Precision	99.97	98	99.46	84.57	0	99.98	99.53	99.54	86.74	0
	Recall	99.63	99.68	96.93	85.72	0	99.96	99.75	97.07	88.27	0.00
	F-score	99.8	98.83	98.17	85.05	0	99.97	99.64	98.29	87.50	0
GRU with two layer	TNR	99.89	99.53	99.99	99.96	100	99.96	99.87	100.00	99.97	100.00
	FPR	0.11	0.47	0.01	0.04	0	0.04	0.13	0.00	0.03	0.00
	FNR	0.38	0.28	2.62	23.67	100	0.05	0.19	2.67	14.96	100.00
	Accuracy	99.67	99.56	99.97	99.9	99.99	99.95	99.86	99.97	99.93	99.99
	Precision	99.98	97.87	99.2	86.37	0	99.99	99.43	99.64	87.35	0
	Recall	99.62	99.72	97.38	76.33	0	99.95	99.81	97.33	85.04	0.00
	F-score	99.8	98.78	98.28	76.73	0	99.97	99.62	98.47	86.18	0