Comparative Study on Heart Disease Prediction Using Feature Selection Techniques on Classification Algorithms

Table 6

Performance measure of best-reduced feature subsets using the filter, wrapper, and embedded methods.

Feature selection techniques	Number of features	Precision						Sensitivity						F-measure
		RF	SVM	DT	KNN	LR	GNB	RF	SVM	DT	KNN	LR	GNB	RF	SVM	DT	KNN	LR	GNB
IFS	13	42.85	35.00	50.00	58.33	35.71	30.00	34.61	26.92	38.46	53.84	38.46	11.53	38.29	30.43	43.47	55.99	37.03	16.66
ANOVA	6	80.95	89.47	71.42	88.23	89.47	68.75	65.38	65.38	57.69	57.69	65.38	42.30	72.34	75.55	63.82	69.76	75.55	52.38
	7	84.21	89.47	69.56	87.50	90.00	68.75	61.53	65.38	61.53	53.84	69.23	42.30	71.11	75.55	65.30	66.66	78.26	52.38
	8	94.44	89.47	68.18	93.33	90.00	70.58	65.38	65.38	57.69	53.84	69.23	46.15	77.27	75.55	62.49	68.29	78.26	55.81
	9	80.95	89.47	70.83	68.75	85.71	70.58	65.38	65.38	65.38	42.30	69.23	46.15	72.34	75.55	68.00	52.38	76.59	55.81
	10	85.00	89.47	70.00	70.58	85.00	70.58	65.38	65.38	53.84	46.15	65.38	46.15	73.91	75.55	60.86	55.81	73.91	55.81
Chi	6	71.42	80.00	64.70	59.25	80.95	56.25	57.69	61.53	42.30	61.53	65.38	34.61	63.82	69.56	51.16	60.37	72.34	42.85
	7	71.42	80.00	61.53	51.85	77.27	60.00	57.69	61.53	61.53	53.84	65.38	46.15	63.82	69.56	61.53	52.83	70.83	52.17
	8	77.27	77.27	68.18	56.66	72.72	57.89	65.38	65.38	57.69	65.38	61.53	42.30	70.83	70.83	62.49	60.71	66.66	48.88
	9	84.21	84.21	82.60	58.62	76.19	57.89	61.53	61.53	73.07	65.38	61.53	42.30	71.11	71.11	77.55	61.81	68.08	48.80
	10	84.21	84.21	82.60	58.62	77.27	57.89	61.53	61.53	73.07	65.38	65.38	42.30	71.11	71.11	77.55	61.81	70.83	48.80
MI	6	85.00	89.47	71.42	88.23	89.47	68.75	65.38	65.38	57.69	57.69	65.38	42.30	73.91	75.55	63.82	69.76	75.55	52.38
	7	88.88	89.47	69.56	87.50	90.00	68.75	61.53	65.38	61.53	53.84	69.23	42.30	72.72	75.55	65.30	66.66	78.26	52.38
	8	80.00	85.00	66.66	59.25	85.71	52.94	61.53	65.38	53.84	61.53	69.23	34.61	69.56	73.91	59.57	60.37	76.59	41.86
	9	85.00	85.00	64.00	59.25	85.71	52.94	65.38	65.38	61.53	61.53	69.23	34.61	73.91	73.91	62.74	60.37	76.59	41.86
	10	80.95	85.00	69.23	59.25	81.81	52.94	65.38	65.38	69.23	61.53	69.23	34.61	72.34	73.91	69.23	60.37	75.00	41.86
Relief	6	84.21	89.47	75.00	72.72	85.00	83.33	61.53	65.38	57.69	61.53	65.38	57.69	71.11	75.55	65.21	66.66	73.91	68.18
	7	88.88	89.47	69.56	87.50	90.00	68.75	61.53	65.38	61.53	53.84	69.23	42.30	72.72	75.55	65.30	66.66	78.26	52.38
	8	94.11	89.47	72.72	93.33	90.00	70.58	61.53	65.38	61.53	53.84	69.23	46.15	74.41	75.55	66.66	68.29	78.26	55.81
	9	83.33	89.47	68.18	68.75	85.71	70.58	57.69	65.38	57.69	42.30	69.23	46.15	68.18	75.55	62.49	52.38	76.59	55.81
	10	85.00	89.47	72.72	70.58	85.00	70.58	65.38	65.38	61.53	46.15	65.38	46.15	73.91	75.55	66.66	55.81	73.91	55.81
FFS	4	83.60	82.35	85.00	88.88	82.35	81.28	73.07	53.84	65.38	61.53	53.84	50.00	79.16	65.11	73.91	72.72	65.11	61.90
	5	86.95	82.35	84.00	88.88	84.21	81.25	76.92	53.84	80.76	61.53	61.53	50.00	81.63	65.11	82.35	72.72	71.11	61.90
	6	86.36	89.47	78.57	63.63	90.00	70.58	73.07	65.38	84.61	53.84	69.23	46.15	79.16	75.55	81.48	58.33	78.26	55.81
	7	80.00	84.21	68.18	50.00	85.71	60.00	61.53	61.53	57.69	53.84	69.23	46.15	69.56	71.11	64.49	51.85	76.59	52.17
	9	88.88	85.00	90.47	58.62	80.95	57.89	61.53	65.38	73.07	65.38	65.38	42.30	72.72	73.91	80.85	61.81	71.34	48.88
BFS	6	90.90	89.47	91.30	90.90	81.81	94.44	76.92	65.38	80.76	76.92	69.23	65.38	83.33	75.55	85.71	83.33	75.00	77.27
	8	84.21	88.23	70.00	56.66	77.27	55.00	61.53	57.69	53.84	65.38	65.38	42.30	71.11	69.76	60.86	60.71	70.83	47.82
	9	87.50	84.21	73.68	50.00	73.91	73.91	53.84	61.53	53.84	53.84	65.38	65.38	66.66	71.11	62.22	51.85	69.38	69.38
	9	88.88	84.21	90.40	58.62	85.71	57.89	61.53	61.53	73.07	65.38	69.23	42.30	72.72	71.11	80.85	61.81	76.59	48.88
	9	88.88	84.21	90.40	58.62	85.71	57.89	61.53	61.53	73.07	65.38	69.23	42.30	72.72	71.11	80.85	61.81	76.59	48.88
EFS	3	85.71	85.71	85.71	85.71	85.71	93.33	69.23	69.23	69.23	69.23	69.23	53.84	76.59	76.59	76.59	76.59	76.59	68.29
	4	75.00	89.47	81.81	80.95	90.00	86.66	69.23	65.38	69.23	65.38	69.23	50.00	71.99	75.55	75.00	72.34	78.26	63.41
	6	80.95	84.21	73.91	59.25	78.26	55.55	65.38	61.53	65.38	61.53	69.23	38.46	72.34	71.11	69.38	60.37	73.46	45.45
	7	85.71	84.21	76.00	59.25	85.71	52.94	69.23	61.53	73.07	61.53	69.23	34.61	76.59	71.11	74.50	60.37	76.59	41.86
	10	85.00	84.21	78.26	58.62	75.00	57.89	65.38	61.53	69.23	65.38	69.23	42.30	80.32	71.11	73.46	61.81	71.99	48.88
RFE	5	81.81	75.00	90.00	90.00	79.16	65.21	69.23	57.69	69.23	69.23	73.07	57.69	75.00	65.21	78.26	78.26	76.00	61.22
	6	78.26	89.47	81.81	81.81	89.47	94.44	69.23	65.38	69.23	69.23	65.38	65.38	73.46	75.55	75.00	75.00	75.55	77.27
	7	86.36	89.47	86.36	94.73	81.81	89.47	73.07	65.38	73.07	69.23	69.23	65.38	79.16	75.55	79.16	79.99	75.00	75.55
	8	94.73	89.47	85.00	85.71	85.00	85.00	69.23	65.38	65.38	69.23	65.38	65.38	79.90	75.55	73.91	76.59	73.91	73.91
	9	84.21	89.47	80.95	90.00	85.00	85.00	61.53	65.38	65.38	69.23	65.38	65.38	71.11	75.55	72.34	78.26	73.91	73.91
Lasso	11	84.21	85.00	82.60	58.62	81.81	57.89	61.53	65.38	73.07	65.38	69.23	42.30	71.11	73.91	77.55	61.81	75.00	48.88
Ridge	5	83.33	69.23	82.60	82.60	65.51	66.66	76.92	69.23	73.07	73.07	73.07	61.53	80.00	69.23	77.55	77.55	69.09	64.00

IFS: initial feature set, ANOVA: ANOVA method, CHI: Chi-square, MI: mutual information, RELIEF: ReliefF method, FFS: forward feature selection, BFS: backward feature selection, EFS: exhaustive feature selection, RFE: recursive feature selection, Lasso: Lasso regression, Ridge: ridge regression.