Trust in Intrusion Detection Systems: An Investigation of Performance Analysis for Machine Learning and Deep Learning Models

Table 14

The performance of machine learning for KDD dataset using all features.

Models	Evaluation metric	Cross-validation performance					Testing performance
		DOS	Normal	Probe	R2L	U2R	DOS	Normal	Probe	R2L	U2R
DT	TNR	99.97	99.98	99.99	100	100	99.95	99.98	100	99.99	100
	FPR	0.03	0.02	0.01	0	0	0.05	0.02	0	0.01	0
	FNR	0	0.08	0.68	3.01	25.83	0	0.07	0.72	5.63	50
	Accuracy	99.99	99.97	99.99	99.99	99.99	99.99	99.97	99.99	99.98	99.99
	Precision	99.99	99.93	99.27	98.1	74.17	99.99	99.89	100	97.1	66.67
	Recall	100	99.92	99.32	96.99	74.17	100	99.93	99.28	94.37	50
	F-score	100	99.92	99.29	97.53	70.6	99.99	99.91	99.64	95.71	57.14
KNN	TNR	99.61	99.94	99.98	99.99	100	99.55	99.89	99.96	99.99	100.00
	FPR	0.39	0.06	0.02	0.01	0	0.45	0.11	0.04	0.01	0.00
	FNR	0.01	0.18	9.76	5.65	100	0.04	0.29	12.27	8.45	100.00
	Accuracy	99.91	99.92	99.9	99.98	99.99	99.89	99.85	99.85	99.97	99.99
	Precision	99.91	99.71	97.82	96.31	0	99.90	99.47	95.67	95.59	0
	Recall	99.99	99.82	90.24	94.35	0	99.96	99.71	87.73	91.55	0.00
	F-score	99.95	99.76	93.87	95.28	0	99.93	99.59	91.53	93.53	0
RF	TNR	99.99	99.98	100	100	100	100.0	99.98	100.00	100.00	100.00
	FPR	0.01	0.02	0	0	0	0.0	0.02	0.00	0.00	0.00
	FNR	0	0.02	0.41	3.23	40	0.0	0.01	0.80	3.52	41.67
	Accuracy	100	99.98	100	99.99	100	100.0	99.98	99.99	99.99	100.00
	Precision	100	99.9	99.97	99.36	93.75	100.0	99.89	99.82	99.70	87.50
	Recall	100	99.98	99.59	96.77	60	100.0	99.99	99.20	96.48	58.33
	F-score	100	99.94	99.78	98.04	77.8	100.0	99.94	99.51	98.06	70.00
NB	TNR	73.23	99.87	97.52	99.02	97.88	71.71	99.86	99.54	99.10	97.89
	FPR	26.77	0.13	2.48	0.98	2.12	28.29	0.14	0.46	0.90	2.11
	FNR	3.07	40.3	93.04	98.65	45	0.63	41.05	100.00	100.00	75.00
	Accuracy	92.44	92.72	96.74	98.77	97.88	94.17	92.65	98.65	98.87	97.88
	Precision	93.93	99	0.3	0.25	0.24	93.82	98.92	0.00	0.00	0.15
	Recall	96.93	59.7	6.96	1.35	55	99.37	58.95	0.00	0.00	25.00
	F-score	95.26	74.48	5.7	4.22	0.6	96.52	73.87	0	0	0.31