Linear Twin Quadratic Surface Support Vector Regression

Table 4

Results of Ridge Regression (I), Linear Regression (II), nu-SVR (III), ϵ-SVR (IV), LSSVR (V), TSVR (VI), LTSVR (VII), and LTQSSVR (VIII) on benchmark data sets with RBF kernel.

Data sets	Alg	RMSE	MAE	SSE/SST	SSR/SST	Time(s)	c	ϵ	σ
Hayes roth; 132 × 6	I	0.6253 ± 0.0601	0.5191 ± 0.0515	0.6914 ± 0.1324	1.0377 ± 0.1817	0.005
	II	0.5495 ± 0.0570	0.4486 ± 0.0487	0.5335 ± 0.1090	0.5345 ± 0.1049	0.0042
	III	1.2221 ± 1.1811	0.7785 ± 0.6059	4.2785 ± 7.4431	4.4979 ± 7.2225	5.8376	2^3	0.001	2^–4
	IV	2.5118 ± 3.8211	1.2269 ± 1.5683	33.4225 ± 69.4087	33.8838 ± 70.3782	0.8978	2^2	0.001	2^–4
	V	0.5787 ± 0.0870	0.4394 ± 0.0832	0.4919 ± 0.1487	0.6280 ± 0.1470	4.0312	2^5		2^2
	VI	0.6157 ± 0.1650	0.4700 ± 0.0966	0.6656 ± 0.4210	1.0015 ± 0.7287	74.9323	2^–4	0.2	2^5
	VII	0.5999 ± 0.1008	0.4450 ± 0.0773	0.5866 ± 0.1730	0.9682 ± 0.3809	208.5302	2^–3	0.01	2^5
	VIII	0.5665 ± 0.0715	0.4394 ± 0.0696	0.5219 ± 0.1174	0.7902 ± 0.2385	6.7313	2^2	0.2
Oring; 23 × 5	I	3.7095 ± 1.5179	2.9936 ± 1.2695	0.5474 ± 0.7138	0.9454 ± 0.5311	0.0026
	II	3.7096 ± 1.5179	2.9936 ± 1.2695	0.5474 ± 0.7138	0.9455 ± 0.5311	0.0086
	III	40.3245 ± 65.7156	26.3581 ± 43.9478	166.4272 ± 367.8244	166.9181 ± 374.4771	0.7148	2^4	0.001	2^0
	IV	511.2011 ± 1.4669e + 03	245.0591 ± 680.3442	7.0421e + 04 ± 2.2234e + 05	7.0542e + 04 ± 2.2272e + 05	0.3195	2^5	0.001	2^–3
	V	4.7860 ± 1.8800	3.8188 ± 1.3995	0.8257 ± 1.0693	0.8409 ± 0.7116	1.6519	2^1		2^–2
	VI	22.4245 ± 44.2676	13.8426 ± 25.2836	46.6614 ± 135.2396	49.9042 ± 143.4053	15.3404	2^3	0.2	2^–2
	VII	28.9964 ± 74.4129	20.1901 ± 50.8615	92.7013 ± 289.5774	90.7100 ± 282.2787	28.7152	2^–3	0.01	2^3
	VIII	5.3305 ± 1.9512	4.1148 ± 1.6144	1.0038 ± 1.3476	1.4496 ± 0.7396	1.5079	2^1	0.2
Container; 15 × 3	I	0.3012 ± 0.0677	0.2792 ± 0.0522	9.2105 ± 5.1356	6.3104 ± 5.0631	0.0027
	II	0.1787 ± 0.0254	0.1574 ± 0.0279	3.1271 ± 1.3823	1.8975 ± 1.6807	0.0068
	III	0.2468 ± 0.3123	0.2143 ± 0.2597	1.8268 ± 0.8391	0.8461 ± 0.8428	0.1491	2^4	0.001	2^–4
	IV	0.1541 ± 0.0395	0.1359 ± 0.0370	2.5484 ± 1.2829	1.2351 ± 1.0699	0.2179	2^–4	0.001	2^5
	V	0.1458 ± 0.0542	0.1343 ± 0.0514	2.1108 ± 1.1380	1.6074 ± 0.9613	1.4901	2^–4		2^5
	VI	0.3147 ± 0.2579	0.2645 ± 0.1928	8.2127 ± 12.0510	6.3811 ± 11.9280	14.4593	2^–3	0.2	2^1
	VII	0.5611 ± 1.0092	0.4895 ± 0.8664	44.1796 ± 114.0210	44.5070 ± 119.0015	16.4264	2^–4	0.001	2^2
	VIII	0.2567 ± 0.1222	0.2277 ± 0.1040	6.2074 ± 4.6990	3.5947 ± 3.4960	0.2904	2^–2	0.1
Daily de; 60 × 13	I	28.1321 ± 4.1695	22.2524 ± 3.1676	0.2260 ± 0.1701	1.2616 ± 0.3249	0.0033
	II	9.7112e − 14 ± 2.0289e − 14	8.1476e − 14 ± 1.9473e − 14	2.8124e − 30 ± 2.2297e − 30	1 ± 8.4956e − 16	0.0022
	III	46.5750 ± 18.2514	26.3692 ± 9.0850	0.2679 ± 0.1062	0.5115 ± 0.2421	0.2335	2^5	0.001	2^–1
	IV	42.4564 ± 13.7516	23.9045 ± 7.3832	0.2352 ± 0.0667	0.4670 ± 0.1249	0.2396	2^5	0.001	2^–1
	V	14.4057 ± 7.2369	9.2370 ± 4.5731	0.0292 ± 0.0176	0.8718 ± 0.1066	2.3495	2^5		2^1
	VI	2.4664 ± 2.1341	1.5724 ± 0.9919	9.5111e − 04 ± 0.0013	0.9875 ± 0.0239	40.7319	2^–4	0.1	2^2
	VII	0.0782 ± 0.0778	0.0486 ± 0.0355	9.4084e − 07 ± 1.5290e − 06	0.9999 ± 5.0720e − 04	32.5313	2^–4	0.2	2^5
	VIII	10.8529 ± 7.5422	6.9757 ± 3.9298	0.0232 ± 0.0274	1.0648 ± 0.1351	9.7285	2^3	0.001
Haber man; 306 × 4	I	0.5854 ± 0.0482	0.4515 ± 0.0431	1.8093 ± 0.3646	1.1981 ± 0.2128	0.0249
	II	0.4259 ± 0.0210	0.3564 ± 0.0123	0.9495 ± 0.1062	0.0889 ± 0.0590	0.0169
	III	0.4877 ± 0.0236	0.4818 ± 0.0195	1.1689 ± 0.1149	0.1572 ± 0.0958	13.646	2^–1	0.001	2^5
	IV	0.4592 ± 0.0262	0.3539 ± 0.0491	1.1258 ± 0.1258	0.2377 ± 0.1436	7.5302	2^–1	0.001	2^3
	V	0.4176 ± 0.0236	0.3422 ± 0.0181	0.9156 ± 0.0695	0.1285 ± 0.0547	33.9049	2^–1		2^–4
	VI	0.4655 ± 0.1198	0.3290 ± 0.0410	1.1776 ± 0.6418	0.4589 ± 0.8390	301.7816	2^–2	0.1	2^2
	VII	0.3273 ± 0.2326	0.3129 ± 0.0410	1.2283 ± 0.2454	0.7573 ± 0.3473	6716.377	2^–3	0.01	2^4
	VIII	0.4446 ± 0.0454	0.3077 ± 0.0310	1.0407 ± 0.1664	0.3038 ± 0.1428	112.8566	2^2	0.1
Wis; 194 × 30	I	0.0088 ± 0.0015	0.0067 ± 7.9729e − 04	0.2061 ± 0.0867	1.2493 ± 0.1829	0.0098
	II	0.0062 ± 0.0017	0.0043 ± 8.0280e − 04	0.1050 ± 0.0512	1.0385 ± 0.1263	0.0029
	III	0.0091 ± 0.0029	0.0057 ± 0.0012	0.1792 ± 0.0688	0.6678 ± 0.1592	8.7957	2^2	0.001	2^–4
	IV	0.0484 ± 0.0048	0.0453 ± 0.0052	6.8505 ± 3.3307	5.8505 ± 3.3307	0.1361	2^–4	0.001	2^–4
	V	0.0415 ± 0.0123	0.0277 ± 0.0061	0.1043 ± 0.0622	0.8787 ± 0.2353	15.4689	2^5		2^2
	VI	0.0057 ± 0.0013	0.0041 ± 6.9250e − 04	0.0768 ± 0.0250	0.9603 ± 0.1223	151.3218	2^–4	0.2	2^3
	VII	0.0060 ± 0.0013	0.0044 ± 6.0337e − 04	0.0858 ± 0.0274	1.0015 ± 0.1607	877.5655	2^–4	0.1	2^4
	VIII	0.0087 ± 0.0016	0.0057 ± 9.3220e − 04	0.1977 ± 0.1165	1.1763 ± 0.2253	83.3716	2^1	0.001
Auto mpg; 392 × 8	I	0.6124 ± 0.0654	0.4813 ± 0.0568	0.5958 ± 0.1036	0.5781 ± 0.1547	0.0067
	II	0.5898 ± 0.0655	0.4632 ± 0.0588	0.5520 ± 0.0947	0.5221 ± 0.1290	0.0018
	III	2.1847 ± 2.5549	1.1657 ± 1.2357	65.4009 ± 126.7810	65.9644 ± 128.8542	36.0491	2^0	0.001	2^5
	IV	1.0737 ± 2.2710	0.6030 ± 1.1399	35.8631 ± 110.4902	36.6985 ± 112.1304	7.6174	2^3	0.001	2^0
	V	0.5598 ± 0.0647	0.3760 ± 0.0577	0.5070 ± 0.0846	0.5995 ± 0.1304	67.7641	2^4		2^–2
	VI	0.5545 ± 0.0367	0.3910 ± 0.0373	0.4989 ± 0.0681	0.6271 ± 0.1919	636.534	2^–4	0.2	2^1
	VII	0.5746 ± 0.0513	0.3942 ± 0.0512	0.5373 ± 0.0993	0.5703 ± 0.1899	10958.5977	2^0	0.001	2^4
	VIII	0.5516 ± 0.0311	0.3774 ± 0.0304	0.4948 ± 0.0715	0.6022 ± 0.1469	90.1874	2^4	0.1
Real estate; 414 × 7	I	9.2684 ± 1.9122	6.7511 ± 0.7482	0.5151 ± 0.1388	0.6653 ± 0.1543	0.003
	II	8.5631 ± 1.8673	6.1966 ± 0.6894	0.4393 ± 0.1278	0.5793 ± 0.1340	0.0019
	III	22.8803 ± 17.2117	13.6317 ± 9.2799	3.9393 ± 6.2263	3.9543 ± 5.7283	65.1059	2^5	0.001	2^1
	IV	0.2448 ± 0.1747	0.1351 ± 0.0448	6.4868 ± 11.0001	6.7479 ± 9.9919	5.6065	2^4	0.001	2^–1
	V	8.3322 ± 2.0481	5.4289 ± 0.5484	0.3697 ± 0.1331	0.5995 ± 0.1285	41.1297	2^2		2^0
	VI	8.2777 ± 2.1514	5.4656 ± 0.6343	0.3653 ± 0.1393	0.6563 ± 0.1276	629.299	2^–4	0.01	2^2
	VII	8.3537 ± 1.9192	5.4032 ± 0.5786	0.3707 ± 0.1209	0.1209 ± 0.1497	10178.2897	2^5	0.001	2^3
	VIII	8.3202 ± 2.0455	5.5315 ± 0.6919	0.3685 ± 0.1295	0.6650 ± 0.1121	52.4031	2^2	0.001
Slump; 103 × 8	I	2.5807 ± 0.5759	2.0662 ± 0.4485	0.1365 ± 0.0793	0.8919 ± 0.2365	0.003
	II	2.5680 ± 0.4819	2.0623 ± 0.3818	0.1324 ± 0.0650	0.8981 ± 0.1997	0.0021
	III	0.0195 ± 0.0058	0.0146 ± 0.0041	0.0161 ± 0.0134	0.9754 ± 0.0510	3.7786	2^5	0.001	2^–2
	IV	0.0678 ± 0.0168	0.0533 ± 0.0113	0.1505 ± 0.0494	0.5995 ± 0.1064	0.6752	2^5	0.001	2^–4
	V	1.4282 ± 0.4010	1.0612 ± 0.3002	0.0380 ± 0.0290	0.8541 ± 0.0565	2.7774	2^5		2^0
	VI	0.7027 ± 0.2838	0.5388 ± 0.1588	0.0097 ± 0.0111	0.9834 ± 0.0584	90.5473	2^–4	0.01	2^1
	VII	0.8015 ± 0.2867	0.5805 ± 0.1746	0.0123 ± 0.0112	0.9997 ± 0.0808	116.1572	2^5	0.1	2^2
	VIII	0.7169 ± 0.1260	0.5188 ± 0.0829	0.0087 ± 0.0031	1.0151 ± 0.0682	8.1602	2^4	0.01
Ozone; 517 × 11	I	4.5653 ± 1.2146	3.5805 ± 0.6836	2.3044 ± 1.2403	1.5985 ± 1.2093	0.0026
	II	2.6971 ± 0.5073	2.1701 ± 0.4575	0.7660 ± 0.1714	0.6772 ± 0.3418	0.0018
	III	2.6364 ± 7.7191	1.7112 ± 4.9129	1.8606e + 03 ± 5.8801e + 03	1.8509e + 03 ± 5.8503e + 03	2.9356	2^3	0.001	2^–4
	IV	80.2195 ± 253.1354	56.6820 ± 178.8180	2.0582e + 06 ± 6.5086e + 06	2.0578e + 06 ± 6.5074e + 06	1.6641	2^5	0.001	2^–4
	V	2.9800 ± 0.4769	2.3844 ± 0.4648	0.8916 ± 0.2495	0.3908 ± 0.2312	5.8676	2^5		2^–1
	VI	2.6906 ± 0.3238	2.1287 ± 0.3460	0.7354 ± 0.2126	0.4839 ± 0.1758	90.8887	2^–4	0.01	2^1
	VII	3.0426 ± 0.6280	2.3599 ± 0.4222	0.9596 ± 0.4674	0.6488 ± 0.3200	118.4467	2^4	0.01	2^2
	VIII	2.8883 ± 0.3950	2.3200 ± 0.3918	0.8525 ± 0.2914	0.5106 ± 0.2615	3.7969	2^2	0.1
Body fat; 252 × 15	I	0.0671 ± 0.0287	0.0446 ± 0.0063	13.3892 ± 11.8766	12.7669 ± 13.0222	0.006
	II	0.0033 ± 0.0022	0.0011 ± 2.3305e − 04	0.0359 ± 0.0342	0.9539 ± 0.0750	0.0044
	III	0.0050 ± 0.0016	0.0025 ± 9.1624e − 04	0.0773 ± 0.0398	0.9214 ± 0.0724	9.4223	2^4	0.001	2^–3
	IV	0.0196 ± 0.0014	0.0160 ± 0.0010	1.1480 ± 0.1444	0.1480 ± 0.1444	0.6526	2^–4	0.001	2^–4
	V	0.0027 ± 0.0014	0.0013 ± 3.6086e − 04	0.0246 ± 0.0289	0.9323 ± 0.0683	25.3603	2^5		2^1
	VI	0.0319 ± 0.0013	0.0316 ± 0.0011	2.7861 ± 0.3020	3.6795 ± 0.3559	2206530	2^–4	0.001	2^1
	VII	0.0026 ± 0.0021	9.7435e − 04 ± 2.8288e − 04	0.0273 ± 0.0388	0.9893 ± 0.0613	1041.3736	2^2	0.1	2^4
	VIII	0.0031 ± 0.0030	8.3023e − 04 ± 4.7873e − 04	0.0507 ± 0.1011	1.0414 ± 0.2066	513.7258	2^1	0.001
Computer; 209 × 8	I	44.1936 ± 14.5749	22.2690 ± 5.1415	0.0788 ± 0.0440	0.9084 ± 0.3800	0.0033
	II	41.8599 ± 13.6174	23.2696 ± 3.8787	0.0725 ± 0.0424	0.9786 ± 0.4010	0.0027
	III	112.3888 ± 133.1836	50.4208 ± 83.6521	5.3675 ± 15.9797	5.7851 ± 15.5227	10.5498	2^5	0.001	2^–1
	IV	1.5487e + 04 ± 8.5220e + 03	5767 ± 3212	1.5393e + 04 ± 1.1253e + 04	1.5350e + 04 ± 1.1198e + 04	0.6358	2^3	0.001	2^–4
	V	51.2335 ± 60.1664	15.0893 ± 15.4565	0.1271 ± 0.1760	0.8189 ± 0.3210	18.4958	2^5		2^0
	VI	28.3684 ± 46.4711	7.4601 ± 9.2279	0.0490 ± 0.1161	0.9014 ± 0.1963	251.1328	2^–4	0.01	2^0
	VII	14.2389 ± 18.4680	4.8588 ± 4.3811	0.0137 ± 0.0253	0.9908 ± 0.1313	750.8074	2^5	0.01	2^1
	VIII	11.1153 ± 17.0344	4.3034 ± 4.3046	0.0097 ± 0.0213	0.9701 ± 0.1138	20.4763	2^2	0.001