Trust in Intrusion Detection Systems: An Investigation of Performance Analysis for Machine Learning and Deep Learning Models

Table 15

The performance of machine learning for KDD dataset using selected features.

Models	Evaluation metric	Cross-validation performance					Testing performance
		Dos	Normal	Probe	R2L	U2R	DOS	Normal	Probe	R2L	U2R
DT	TNR	98.75	99.95	99.98	100	100	98.91	99.95	99.97	99.99	100.00
	FPR	1.25	0.05	0.02	0	0	1.09	0.05	0.03	0.01	0.00
	FNR	0.06	0.17	25.37	2.4	50.83	0.06	0.18	22.76	2.05	75.00
	Accuracy	99.72	99.93	99.76	99.99	99.99	99.74	99.93	99.78	99.99	99.99
	Precision	99.71	99.76	96.75	98.43	75.93	99.74	99.77	95.60	97.95	42.86
	Recall	99.94	99.86	74.63	97.6	49.17	99.94	99.82	77.24	97.95	25.00
	F-score	99.83	99.8	84.25	98	57.78	99.84	99.80	85.45	97.95	31.58
KNN	TNR	98.55	99.87	99.97	99.99	100	98.72	99.86	99.98	99.99	100.00
	FPR	1.45	0.13	0.03	0.01	0	1.28	0.14	0.02	0.01	0.00
	FNR	0.07	0.38	27.65	16.11	100	0.07	0.33	25.07	19.94	100.00
	Accuracy	99.67	99.83	99.73	99.95	99.99	99.70	99.83	99.76	99.94	99.99
	Precision	99.66	99.41	95.89	96.8	0.0	99.70	99.37	96.45	94.79	0.0
	Recall	99.93	99.62	72.35	83.89	0	99.93	99.67	74.93	80.06	0.00
	F-score	99.8	99.51	82.46	89.81	0.0	99.81	99.52	84.34	86.80	0.0
RF	TNR	98.76	99.94	99.98	100	100	98.91	99.95	99.97	100.00	100.00
	FPR	1.24	0.06	0.02	0	0	1.09	0.05	0.03	0.00	0.00
	FNR	0.06	0.14	25.29	3.08	40	0.06	0.13	22.31	3.81	58.33
	Accuracy	99.72	99.93	99.76	99.99	100	99.74	99.94	99.78	99.99	99.99
	Precision	99.71	99.76	96.66	99.74	87.04	99.74	99.77	96.15	99.39	71.43
	Recall	99.94	99.86	74.71	96.92	60	99.94	99.87	77.69	96.19	41.67
	F-score	99.83	99.8	84.26	98.3	71.85	99.84	99.82	85.94	97.76	52.63
NB	TNR	21.6	99.75	97.65	99.49	79.47	21.27	99.72	99.72	99.52	77.36
	FPR	78.4	0.25	2.35	0.51	20.53	78.73	0.28	0.28	0.48	22.64
	FNR	24.7	97.35	92.97	98.64	20.83	24.90	96.72	100.00	100.00	8.33
	Accuracy	65.13	82.46	96.87	99.24	79.47	64.87	82.48	98.87	99.27	77.36
	Precision	80.44	66.47	0.29	0.48	0.09	80.27	71.77	0.00	0.00	0.04
	Recall	75.3	2.65	7.03	1.36	79.17	75.10	3.28	0.00	0.00	91.67
	F-score	77.78	5.08	5.61	3.56	0.19	77.60	6.27	0	0	0.07