Optimization Benchmarks

Benchmarking Rules

• Seed used: 11111
• Any tolerance value: 1e-20
• SANN extra parameters: maxit=1000000, temp=4, tmax=5
• L-BFGS-B knows the function parameter bounds (provided)
• CG uses Fletcher–Reeves update
• Matching with the expected value has a tolerance of 1e-5

Acronyms

• NM: Nelder-Mead
• BFGS: Broyden-Fletcher-Goldfarb-Shanno
• L-BFGS-B: Limited BFGS Bound
• CG: Conjugate Gradient
• SANN: Simulated Annealing

Comparative Table

Passing test, in bold if it passes:

ID	Name	Successes	NM	BFGS	L-BFGS-B	CG	SANN
01	[Ackley Function]	0
02	[Bukin Function number 6]	0
03	[Cross-In-Tray Function]	1					X
04	[Drop-Wave Function]	2			X	X
05	[Eggholder Function]	0
06	[Gramacy & Lee Function]	0
07	[Griewank Function]	4	X	X	X	X
08	[Holder Table Function]	0
09	[Langermann Function]	0
10	[Levy Function]	5	X	X	X	X	X
11	[Levy Function number 13]	1					X
12	[Rastrigin Function]	5	X	X	X	X	X
13	[Schaffer Function number 2]	5	X	X	X	X	X
14	[Schaffer Function number 4]	0
15	[Schwefel Function]	0
16	[Schubert Function]	1				X
17	[Bohachevsky Function number 1]	5	X	X	X	X	X
18	[Perm Function 0, d=2, beta=10]	5	X	X	X	X	X
19	[Rotated HYper-Ellipsoid Function]	5	X	X	X	X	X
20	[Sphere Function]	5	X	X	X	X	X
21	[Sum of Different Powers Function]	5	X	X	X	X	X
22	[Sum Squares Function]	5	X	X	X	X	X
23	[Trid Function]	0
24	[Booth Function]	5	X	X	X	X	X
25	[Matyas Function]	5	X	X	X	X	X
26	[McCormick Function]	0
27	[Power Sum Function]	0
28	[Zakharov Function]	5	X	X	X	X	X
29	[Three-Hump Camel Function]	5	X	X	X	X	X
30	[Sixth-Hump Camel Function]	0
31	[Dixon-Price Function]	5	X	X	X	X	X
32	[Rosenbrock Function]	4	X	X	X		X
33	[De Jong Function number 5]	0
34	[Easom Function]	1			X
35	[Michalewicz Function]	2			X	X
36	[Beale Function]	4	X	X	X		X
37	[Branin Function]	5	X	X	X	X	X
38	[Colville Function]	2		X	X
39	[Forrester et al. Function]	0
40	[Goldstein-Price Function]	1	X
41	[Hartmann 3-Dimensional Function]	1			X
42	[Hartmann 4-Dimensional Function]	0
43	[Hartmann 6-Dimensional Function]	0
44	[Perm Function d=2, beta=0.5]	4	X	X	X		X
45	[Powell Function]	3		X	X	X
46	[Shekel Function]	0
47	[Styblinski-Tang Function]	0

Predicted value, in bold the closest to the expected value:

ID	Success Rate	Expected	NM	BFGS	L-BFGS-B	CG	SANN
01	000%	0.000000	19.991889	19.987198	19.987198	19.987198	19.900286
02	000%	0.000000	0.062869	0.056204	0.065493	0.055360	0.130773
03	020%	-2.062610	-1.790496	-1.790532	-1.790602	-1.790602	-2.062612
04	040%	-1.000000	-0.997581	-0.999571	-1.000000	-1.000000	-0.997581
05	000%	-959.640700	-28.524986	-42.711521	-66.843717	-66.843717	-66.843717
06	000%	-0.869011	-0.355361	-0.074910	-0.232259	-0.223492	-2.873899
07	080%	0.000000	0.000000	0.000000	0.000000	0.000000	0.003090
08	000%	-19.208500	-0.004513	-0.498161	-1.732972	-36685769350.000000	-5.712245
09	000%	-1.500000	-0.144303	-0.265339	-0.414367	-0.328105	-2.193738
10	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
11	020%	0.000000	1.980921	1.073325	1.980921	1.980921	0.000004
12	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
13	100%	0.000000	0.000000	0.000000	0.000000	0.000001	0.000000
14	000%	0.292579	0.500001	0.540176	0.533777	0.540176	0.500086
15	000%	0.000000	830.075197	830.075197	830.075197	830.075197	830.075197
16	020%	-186.730900	-4.722417	-2.418022	-37.681120	-186.730909	-186.730880
17	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
18	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
19	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
20	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
21	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
22	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
23	000%	-200.000000	-0.168717	-13.949876	-50.000000	-50.000000	-49.994434
24	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
25	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
26	000%	-1.913300	1.228370	1.228370	1.228370	1.228370	-724.479527
27	000%	0.000000	0.000012	0.000043	0.000429	3.906389	0.000437
28	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000
29	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
30	000%	-1.031600	-0.000111	-0.007523	-1.031628	-1.031628	-1.031628
31	100%	0.000000	0.000000	0.000000	0.000000	0.000000	0.000001
32	080%	0.000000	0.000000	0.000000	0.000000	0.374468	0.000003
33	000%	0.998000	13.618609	499.998853	21.988408	499.999987	23.809434
34	020%	-1.000000	0.000000	0.000000	-1.000000	0.000000	-0.997678
35	040%	-1.801300	-1.015766	-1.000949	-1.801303	-1.801303	-1.997497
36	080%	0.000000	0.000000	0.000000	0.000000	0.000036	0.000001
37	100%	0.397887	0.397887	0.397887	0.397887	0.397887	0.397888
38	040%	0.000000	7.837540	0.000000	0.000000	0.004784	0.002068
39	000%	-6.020740	-2.596838	-1379.991187	0.665095	-1379.991187	-510.883737
40	020%	3.000000	3.000000	30.000000	30.000000	30.000000	3.000010
41	020%	-3.862780	-0.857013	-1.891888	-3.862780	-3.862462	-2.927386
42	000%	-3.135474	-1.232411	-1.843535	-2.896422	-3.134494	-1.091476
43	000%	-3.223700	-1.610125	-2.350558	-3.042458	-3.042458	-1.592905
44	080%	0.000000	0.000000	0.000000	0.000000	0.000142	0.000001
45	060%	0.000000	7.734509	0.000000	0.000000	0.000004	0.179920
46	000%	-10.536400	-0.844935	-0.862023	-10.532087	-4.079038	-0.849542
47	000%	-78.331980	-0.002132	-71.718577	-78.332331	-78.332331	-64.195612

Speed comparison, in bold the fastest:

ID	Average Time (SANN-less)	NM	BFGS	L-BFGS-B	CG	SANN
01	0.076075	0.076100	0.075000	0.077100	0.076100	8.066800
02	0.076600	0.080100	0.071100	0.075100	0.080100	5.405800
03	0.077575	0.076100	0.076100	0.082000	0.076100	6.719800
04	0.074825	0.073100	0.081100	0.072000	0.073100	6.214400
05	0.074100	0.073100	0.076100	0.074100	0.073100	6.349500
06	0.073850	0.074100	0.075100	0.072100	0.074100	4.603300
07	0.075575	0.075100	0.077000	0.075100	0.075100	5.554900
08	0.074850	0.075100	0.072100	0.077100	0.075100	6.848900
09	0.077100	0.074100	0.076100	0.084100	0.074100	24.656500
10	0.078600	0.079100	0.080100	0.076100	0.079100	11.024900
11	0.079600	0.073100	0.077100	0.095100	0.073100	8.409000
12	0.075100	0.076100	0.075100	0.073100	0.076100	5.233700
13	0.077825	0.076100	0.075100	0.084000	0.076100	6.504800
14	0.093850	0.082100	0.075100	0.136100	0.082100	7.671400
15	0.073850	0.074100	0.075100	0.072100	0.074100	5.473900
16	0.074300	0.070100	0.083000	0.074000	0.070100	8.331900
17	0.078100	0.076100	0.080100	0.080100	0.076100	6.776800
18	0.080300	0.077000	0.086100	0.081100	0.077000	19.608900
19	0.076075	0.075100	0.080100	0.074000	0.075100	26.520800
20	0.072825	0.072100	0.076100	0.071000	0.072100	3.478500
21	0.076600	0.076100	0.079100	0.075100	0.076100	4.801400
22	0.074550	0.072000	0.079100	0.075100	0.072000	5.250500
23	0.074325	0.072100	0.074100	0.079000	0.072100	8.096800
24	0.074300	0.074000	0.075100	0.074100	0.074000	5.094600
25	0.075325	0.077100	0.073100	0.074000	0.077100	5.085600
26	0.077600	0.076100	0.079100	0.079100	0.076100	7.152100
27	0.115600	0.077100	0.122100	0.186100	0.077100	16.586800
28	0.076825	0.073100	0.083100	0.078000	0.073100	6.402500
29	0.078600	0.073100	0.087100	0.081100	0.073100	6.456600
30	0.087100	0.073100	0.125100	0.077100	0.073100	7.728000
31	0.079600	0.073100	0.091100	0.081100	0.073100	8.280000
32	0.080850	0.078100	0.083100	0.084100	0.078100	7.061000
33	0.089350	0.083100	0.109100	0.082100	0.083100	22.117700
34	0.091850	0.080100	0.097100	0.110100	0.080100	6.022300
35	0.076100	0.075100	0.076100	0.078100	0.075100	6.628700
36	0.080100	0.077100	0.091100	0.075100	0.077100	7.203100
37	0.081600	0.072100	0.105100	0.077100	0.072100	8.543400
38	0.085350	0.081100	0.096100	0.083100	0.081100	10.988100
39	0.081350	0.084100	0.082100	0.075100	0.084100	6.157400
40	0.079600	0.079100	0.081100	0.079100	0.079100	10.963800
41	0.077850	0.072100	0.084100	0.083100	0.072100	24.183200
42	0.085100	0.074100	0.102100	0.090100	0.074100	26.449800
43	0.087350	0.071100	0.111100	0.096100	0.071100	27.222300
44	0.080100	0.075100	0.091100	0.079100	0.075100	20.305400
45	0.307225	0.129100	0.407300	0.563400	0.129100	104.793000
46	0.077600	0.074100	0.080100	0.082100	0.074100	29.616200
47	0.077575	0.077100	0.079000	0.077100	0.077100	4.754400

Images

NM

BFGS

L-BFGS-B

CG

SANN

Text

NM

01/47 in  0.0761 seconds:    19.991889, supposed     0.000000 =>    19.991889 difference [Ackley Function]
02/47 in  0.0801 seconds:     0.062869, supposed     0.000000 =>     0.062869 difference [Bukin Function number 6]
03/47 in  0.0761 seconds:    -1.790496, supposed    -2.062610 =>     0.272114 difference [Cross-In-Tray Function]
04/47 in  0.0731 seconds:    -0.997581, supposed    -1.000000 =>     0.002419 difference [Drop-Wave Function]
05/47 in  0.0731 seconds:   -28.524986, supposed  -959.640700 =>   931.115714 difference [Eggholder Function]
06/47 in  0.0741 seconds:    -0.355361, supposed    -0.869011 =>     0.513650 difference [Gramacy & Lee Function]
07/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Griewank Function]
08/47 in  0.0751 seconds:    -0.004513, supposed   -19.208500 =>    19.203987 difference [Holder Table Function]
09/47 in  0.0741 seconds:    -0.144303, supposed    -1.500000 =>     1.355697 difference [Langermann Function]
10/47 in  0.0791 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Levy Function]
11/47 in  0.0731 seconds:     1.980921, supposed     0.000000 =>     1.980921 difference [Levy Function number 13]
12/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rastrigin Function]
13/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Schaffer Function number 2]
14/47 in  0.0821 seconds:     0.500001, supposed     0.292579 =>     0.207422 difference [Schaffer Function number 4]
15/47 in  0.0741 seconds:   830.075197, supposed     0.000000 =>   830.075197 difference [Schwefel Function]
16/47 in  0.0701 seconds:    -4.722417, supposed  -186.730900 =>   182.008483 difference [Schubert Function]
17/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Bohachevsky Function number 1]
18/47 in  0.0770 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function 0, d=2, beta=10]
19/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rotated HYper-Ellipsoid Function]
20/47 in  0.0721 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sphere Function]
21/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum of Different Powers Function]
22/47 in  0.0720 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum Squares Function]
23/47 in  0.0721 seconds:    -0.168717, supposed  -200.000000 =>   199.831283 difference [Trid Function]
24/47 in  0.0740 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Booth Function]
25/47 in  0.0771 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Matyas Function]
26/47 in  0.0761 seconds:     1.228370, supposed    -1.913300 =>     3.141670 difference [McCormick Function]
27/47 in  0.0771 seconds:     0.000012, supposed     0.000000 =>     0.000012 difference [Power Sum Function]
28/47 in  0.0731 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Zakharov Function]
29/47 in  0.0731 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Three-Hump Camel Function]
30/47 in  0.0731 seconds:    -0.000111, supposed    -1.031600 =>     1.031489 difference [Sixth-Hump Camel Function]
31/47 in  0.0731 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Dixon-Price Function]
32/47 in  0.0781 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rosenbrock Function]
33/47 in  0.0831 seconds:    13.618609, supposed     0.998000 =>    12.620609 difference [De Jong Function number 5]
34/47 in  0.0801 seconds:    -0.000000, supposed    -1.000000 =>     1.000000 difference [Easom Function]
35/47 in  0.0751 seconds:    -1.015766, supposed    -1.801300 =>     0.785534 difference [Michalewicz Function]
36/47 in  0.0771 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Beale Function]
37/47 in  0.0721 seconds:     0.397887, supposed     0.397887 =>     0.000000 difference [Branin Function]
38/47 in  0.0811 seconds:     7.837540, supposed     0.000000 =>     7.837540 difference [Colville Function]
39/47 in  0.0841 seconds:    -2.596838, supposed    -6.020740 =>     3.423902 difference [Forrester et al. Function]
40/47 in  0.0791 seconds:     3.000000, supposed     3.000000 =>     0.000000 difference [Goldstein-Price Function]
41/47 in  0.0721 seconds:    -0.857013, supposed    -3.862780 =>     3.005767 difference [Hartmann 3-Dimensional Function]
42/47 in  0.0741 seconds:    -1.232411, supposed    -3.135474 =>     1.903063 difference [Hartmann 4-Dimensional Function]
43/47 in  0.0711 seconds:    -1.610125, supposed    -3.223700 =>     1.613575 difference [Hartmann 6-Dimensional Function]
44/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function d=2, beta=0.5]
45/47 in  0.1291 seconds:     7.734509, supposed     0.000000 =>     7.734509 difference [Powell Function]
46/47 in  0.0741 seconds:    -0.844935, supposed   -10.536400 =>     9.691465 difference [Shekel Function]
47/47 in  0.0771 seconds:    -0.002132, supposed   -78.331980 =>    78.329848 difference [Styblinski-Tang Function]

0.0767s +-0.0084s average computation time.
Average difference: 49.33491 +- 181.8976.
0 fatal errors.
27 errors.
20 perfect results.

BFGS

01/47 in  0.0750 seconds:    19.987198, supposed     0.000000 =>    19.987198 difference [Ackley Function]
02/47 in  0.0711 seconds:     0.056204, supposed     0.000000 =>     0.056204 difference [Bukin Function number 6]
03/47 in  0.0761 seconds:    -1.790532, supposed    -2.062610 =>     0.272078 difference [Cross-In-Tray Function]
04/47 in  0.0811 seconds:    -0.999571, supposed    -1.000000 =>     0.000429 difference [Drop-Wave Function]
05/47 in  0.0761 seconds:   -42.711521, supposed  -959.640700 =>   916.929179 difference [Eggholder Function]
06/47 in  0.0751 seconds:    -0.074910, supposed    -0.869011 =>     0.794101 difference [Gramacy & Lee Function]
07/47 in  0.0770 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Griewank Function]
08/47 in  0.0721 seconds:    -0.498161, supposed   -19.208500 =>    18.710339 difference [Holder Table Function]
09/47 in  0.0761 seconds:    -0.265339, supposed    -1.500000 =>     1.234661 difference [Langermann Function]
10/47 in  0.0801 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Levy Function]
11/47 in  0.0771 seconds:     1.073325, supposed     0.000000 =>     1.073325 difference [Levy Function number 13]
12/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rastrigin Function]
13/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Schaffer Function number 2]
14/47 in  0.0751 seconds:     0.540176, supposed     0.292579 =>     0.247597 difference [Schaffer Function number 4]
15/47 in  0.0751 seconds:   830.075197, supposed     0.000000 =>   830.075197 difference [Schwefel Function]
16/47 in  0.0830 seconds:    -2.418022, supposed  -186.730900 =>   184.312878 difference [Schubert Function]
17/47 in  0.0801 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Bohachevsky Function number 1]
18/47 in  0.0861 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function 0, d=2, beta=10]
19/47 in  0.0801 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rotated HYper-Ellipsoid Function]
20/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sphere Function]
21/47 in  0.0791 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum of Different Powers Function]
22/47 in  0.0791 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum Squares Function]
23/47 in  0.0741 seconds:   -13.949876, supposed  -200.000000 =>   186.050124 difference [Trid Function]
24/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Booth Function]
25/47 in  0.0731 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Matyas Function]
26/47 in  0.0791 seconds:     1.228370, supposed    -1.913300 =>     3.141670 difference [McCormick Function]
27/47 in  0.1221 seconds:     0.000043, supposed     0.000000 =>     0.000043 difference [Power Sum Function]
28/47 in  0.0831 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Zakharov Function]
29/47 in  0.0871 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Three-Hump Camel Function]
30/47 in  0.1251 seconds:    -0.007523, supposed    -1.031600 =>     1.024077 difference [Sixth-Hump Camel Function]
31/47 in  0.0911 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Dixon-Price Function]
32/47 in  0.0831 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rosenbrock Function]
33/47 in  0.1091 seconds:   499.998853, supposed     0.998000 =>   499.000853 difference [De Jong Function number 5]
34/47 in  0.0971 seconds:    -0.000000, supposed    -1.000000 =>     1.000000 difference [Easom Function]
35/47 in  0.0761 seconds:    -1.000949, supposed    -1.801300 =>     0.800351 difference [Michalewicz Function]
36/47 in  0.0911 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Beale Function]
37/47 in  0.1051 seconds:     0.397887, supposed     0.397887 =>     0.000000 difference [Branin Function]
38/47 in  0.0961 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Colville Function]
39/47 in  0.0821 seconds: -1379.991187, supposed    -6.020740 =>  1373.970447 difference [Forrester et al. Function]
40/47 in  0.0811 seconds:    30.000000, supposed     3.000000 =>    27.000000 difference [Goldstein-Price Function]
41/47 in  0.0841 seconds:    -1.891888, supposed    -3.862780 =>     1.970892 difference [Hartmann 3-Dimensional Function]
42/47 in  0.1021 seconds:    -1.843535, supposed    -3.135474 =>     1.291939 difference [Hartmann 4-Dimensional Function]
43/47 in  0.1111 seconds:    -2.350558, supposed    -3.223700 =>     0.873142 difference [Hartmann 6-Dimensional Function]
44/47 in  0.0911 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function d=2, beta=0.5]
45/47 in  0.4073 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Powell Function]
46/47 in  0.0801 seconds:    -0.862023, supposed   -10.536400 =>     9.674377 difference [Shekel Function]
47/47 in  0.0790 seconds:   -71.718577, supposed   -78.331980 =>     6.613403 difference [Styblinski-Tang Function]

0.0912s +-0.0488s average computation time.
Average difference: 86.93839 +- 271.1315.
0 fatal errors.
26 errors.
21 perfect results.

L-BFGS-B

01/47 in  0.0771 seconds:    19.987198, supposed     0.000000 =>    19.987198 difference [Ackley Function]
02/47 in  0.0751 seconds:     0.065493, supposed     0.000000 =>     0.065493 difference [Bukin Function number 6]
03/47 in  0.0820 seconds:    -1.790602, supposed    -2.062610 =>     0.272008 difference [Cross-In-Tray Function]
04/47 in  0.0720 seconds:    -1.000000, supposed    -1.000000 =>     0.000000 difference [Drop-Wave Function]
05/47 in  0.0741 seconds:   -66.843717, supposed  -959.640700 =>   892.796983 difference [Eggholder Function]
06/47 in  0.0721 seconds:    -0.232259, supposed    -0.869011 =>     0.636752 difference [Gramacy & Lee Function]
07/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Griewank Function]
08/47 in  0.0771 seconds:    -1.732972, supposed   -19.208500 =>    17.475528 difference [Holder Table Function]
09/47 in  0.0841 seconds:    -0.414367, supposed    -1.500000 =>     1.085633 difference [Langermann Function]
10/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Levy Function]
11/47 in  0.0951 seconds:     1.980921, supposed     0.000000 =>     1.980921 difference [Levy Function number 13]
12/47 in  0.0731 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rastrigin Function]
13/47 in  0.0840 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Schaffer Function number 2]
14/47 in  0.1361 seconds:     0.533777, supposed     0.292579 =>     0.241198 difference [Schaffer Function number 4]
15/47 in  0.0721 seconds:   830.075197, supposed     0.000000 =>   830.075197 difference [Schwefel Function]
16/47 in  0.0740 seconds:   -37.681120, supposed  -186.730900 =>   149.049780 difference [Schubert Function]
17/47 in  0.0801 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Bohachevsky Function number 1]
18/47 in  0.0811 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function 0, d=2, beta=10]
19/47 in  0.0740 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rotated HYper-Ellipsoid Function]
20/47 in  0.0710 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sphere Function]
21/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum of Different Powers Function]
22/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum Squares Function]
23/47 in  0.0790 seconds:   -50.000000, supposed  -200.000000 =>   150.000000 difference [Trid Function]
24/47 in  0.0741 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Booth Function]
25/47 in  0.0740 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Matyas Function]
26/47 in  0.0791 seconds:     1.228370, supposed    -1.913300 =>     3.141670 difference [McCormick Function]
27/47 in  0.1861 seconds:     0.000429, supposed     0.000000 =>     0.000429 difference [Power Sum Function]
28/47 in  0.0780 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Zakharov Function]
29/47 in  0.0811 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Three-Hump Camel Function]
30/47 in  0.0771 seconds:    -1.031628, supposed    -1.031600 =>     0.000028 difference [Sixth-Hump Camel Function]
31/47 in  0.0811 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Dixon-Price Function]
32/47 in  0.0841 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rosenbrock Function]
33/47 in  0.0821 seconds:    21.988408, supposed     0.998000 =>    20.990408 difference [De Jong Function number 5]
34/47 in  0.1101 seconds:    -1.000000, supposed    -1.000000 =>     0.000000 difference [Easom Function]
35/47 in  0.0781 seconds:    -1.801303, supposed    -1.801300 =>     0.000003 difference [Michalewicz Function]
36/47 in  0.0751 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Beale Function]
37/47 in  0.0771 seconds:     0.397887, supposed     0.397887 =>     0.000000 difference [Branin Function]
38/47 in  0.0831 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Colville Function]
39/47 in  0.0751 seconds:     0.665095, supposed    -6.020740 =>     6.685835 difference [Forrester et al. Function]
40/47 in  0.0791 seconds:    30.000000, supposed     3.000000 =>    27.000000 difference [Goldstein-Price Function]
41/47 in  0.0831 seconds:    -3.862780, supposed    -3.862780 =>     0.000000 difference [Hartmann 3-Dimensional Function]
42/47 in  0.0901 seconds:    -2.896422, supposed    -3.135474 =>     0.239052 difference [Hartmann 4-Dimensional Function]
43/47 in  0.0961 seconds:    -3.042458, supposed    -3.223700 =>     0.181242 difference [Hartmann 6-Dimensional Function]
44/47 in  0.0791 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function d=2, beta=0.5]
45/47 in  0.5634 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Powell Function]
46/47 in  0.0821 seconds:   -10.532087, supposed   -10.536400 =>     0.004313 difference [Shekel Function]
47/47 in  0.0771 seconds:   -78.332331, supposed   -78.331980 =>     0.000351 difference [Styblinski-Tang Function]

0.0932s +-0.0726s average computation time.
Average difference: 45.14702 +- 176.7437.
0 fatal errors.
22 errors.
25 perfect results.

CG

01/47 in  0.0820 seconds:    19.987198, supposed     0.000000 =>    19.987198 difference [Ackley Function]
02/47 in  0.0841 seconds:     0.055360, supposed     0.000000 =>     0.055360 difference [Bukin Function number 6]
03/47 in  0.0791 seconds:    -1.790602, supposed    -2.062610 =>     0.272008 difference [Cross-In-Tray Function]
04/47 in  0.0821 seconds:    -1.000000, supposed    -1.000000 =>     0.000000 difference [Drop-Wave Function]
05/47 in  0.0801 seconds:   -66.843717, supposed  -959.640700 =>   892.796983 difference [Eggholder Function]
06/47 in  0.0791 seconds:    -0.223492, supposed    -0.869011 =>     0.645519 difference [Gramacy & Lee Function]
07/47 in  0.0931 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Griewank Function]
08/47 in  0.0831 seconds: -3668576935063003886426208268822020880240806822882448004860686862420400466466022868086028846846280006628246062222088622468042604002080644642680648200680822208626402448020668464868482246248022824888668886644244800422606240202202664224248008804466402206822040042828886006626842220080608680220822646286068.000000, supposed   -19.208500 => 3668576935063003886426208268822020880240806822882448004860686862420400466466022868086028846846280006628246062222088622468042604002080644642680648200680822208626402448020668464868482246248022824888668886644244800422606240202202664224248008804466402206822040042828886006626842220080608680220822646286068.000000 difference [Holder Table Function]
09/47 in  0.1051 seconds:    -0.328105, supposed    -1.500000 =>     1.171895 difference [Langermann Function]
10/47 in  0.0861 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Levy Function]
11/47 in  0.0831 seconds:     1.980921, supposed     0.000000 =>     1.980921 difference [Levy Function number 13]
12/47 in  0.0861 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rastrigin Function]
13/47 in  0.0791 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Schaffer Function number 2]
14/47 in  0.0821 seconds:     0.540176, supposed     0.292579 =>     0.247597 difference [Schaffer Function number 4]
15/47 in  0.0851 seconds:   830.075197, supposed     0.000000 =>   830.075197 difference [Schwefel Function]
16/47 in  0.0821 seconds:  -186.730909, supposed  -186.730900 =>     0.000009 difference [Schubert Function]
17/47 in  0.0811 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Bohachevsky Function number 1]
18/47 in  0.0971 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Perm Function 0, d=2, beta=10]
19/47 in  0.0881 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rotated HYper-Ellipsoid Function]
20/47 in  0.0821 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sphere Function]
21/47 in  0.0801 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum of Different Powers Function]
22/47 in  0.0821 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum Squares Function]
23/47 in  0.0941 seconds:   -50.000000, supposed  -200.000000 =>   150.000000 difference [Trid Function]
24/47 in  0.0791 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Booth Function]
25/47 in  0.0851 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Matyas Function]
26/47 in  0.0821 seconds:     1.228370, supposed    -1.913300 =>     3.141670 difference [McCormick Function]
27/47 in  0.0931 seconds:     3.906389, supposed     0.000000 =>     3.906389 difference [Power Sum Function]
28/47 in  0.0841 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Zakharov Function]
29/47 in  0.0811 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Three-Hump Camel Function]
30/47 in  0.0771 seconds:    -1.031628, supposed    -1.031600 =>     0.000028 difference [Sixth-Hump Camel Function]
31/47 in  0.0761 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Dixon-Price Function]
32/47 in  0.0761 seconds:     0.374468, supposed     0.000000 =>     0.374468 difference [Rosenbrock Function]
33/47 in  0.0820 seconds:   499.999987, supposed     0.998000 =>   499.001987 difference [De Jong Function number 5]
34/47 in  0.0741 seconds:    -0.000000, supposed    -1.000000 =>     1.000000 difference [Easom Function]
35/47 in  0.0891 seconds:    -1.801303, supposed    -1.801300 =>     0.000003 difference [Michalewicz Function]
36/47 in  0.0761 seconds:     0.000036, supposed     0.000000 =>     0.000036 difference [Beale Function]
37/47 in  0.0881 seconds:     0.397887, supposed     0.397887 =>     0.000000 difference [Branin Function]
38/47 in  0.0851 seconds:     0.004784, supposed     0.000000 =>     0.004784 difference [Colville Function]
39/47 in  0.0731 seconds: -1379.991187, supposed    -6.020740 =>  1373.970447 difference [Forrester et al. Function]
40/47 in  0.0781 seconds:    30.000000, supposed     3.000000 =>    27.000000 difference [Goldstein-Price Function]
41/47 in  0.0941 seconds:    -3.862462, supposed    -3.862780 =>     0.000318 difference [Hartmann 3-Dimensional Function]
42/47 in  0.0961 seconds:    -3.134494, supposed    -3.135474 =>     0.000980 difference [Hartmann 4-Dimensional Function]
43/47 in  0.0921 seconds:    -3.042458, supposed    -3.223700 =>     0.181242 difference [Hartmann 6-Dimensional Function]
44/47 in  0.0861 seconds:     0.000142, supposed     0.000000 =>     0.000142 difference [Perm Function d=2, beta=0.5]
45/47 in  0.4093 seconds:     0.000004, supposed     0.000000 =>     0.000004 difference [Powell Function]
46/47 in  0.0911 seconds:    -4.079038, supposed   -10.536400 =>     6.457362 difference [Shekel Function]
47/47 in  0.0881 seconds:   -78.332331, supposed   -78.331980 =>     0.000351 difference [Styblinski-Tang Function]

0.0913s +-0.0478s average computation time.
Average difference: 82.8755 +- 272.0567.
1 fatal errors.
26 errors.
20 perfect results.

SANN

01/47 in  8.0668 seconds:    19.900286, supposed     0.000000 =>    19.900286 difference [Ackley Function]
02/47 in  5.4058 seconds:     0.130773, supposed     0.000000 =>     0.130773 difference [Bukin Function number 6]
03/47 in  6.7198 seconds:    -2.062612, supposed    -2.062610 =>     0.000002 difference [Cross-In-Tray Function]
04/47 in  6.2144 seconds:    -0.997581, supposed    -1.000000 =>     0.002419 difference [Drop-Wave Function]
05/47 in  6.3495 seconds:   -66.843717, supposed  -959.640700 =>   892.796983 difference [Eggholder Function]
06/47 in  4.6033 seconds:    -2.873899, supposed    -0.869011 =>     2.004888 difference [Gramacy & Lee Function]
07/47 in  5.5549 seconds:     0.003090, supposed     0.000000 =>     0.003090 difference [Griewank Function]
08/47 in  6.8489 seconds:    -5.712245, supposed   -19.208500 =>    13.496255 difference [Holder Table Function]
09/47 in 24.6565 seconds:    -2.193738, supposed    -1.500000 =>     0.693738 difference [Langermann Function]
10/47 in 11.0249 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Levy Function]
11/47 in  8.4090 seconds:     0.000004, supposed     0.000000 =>     0.000004 difference [Levy Function number 13]
12/47 in  5.2337 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Rastrigin Function]
13/47 in  6.5048 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Schaffer Function number 2]
14/47 in  7.6714 seconds:     0.500086, supposed     0.292579 =>     0.207507 difference [Schaffer Function number 4]
15/47 in  5.4739 seconds:   830.075197, supposed     0.000000 =>   830.075197 difference [Schwefel Function]
16/47 in  8.3319 seconds:  -186.730880, supposed  -186.730900 =>     0.000020 difference [Schubert Function]
17/47 in  6.7768 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Bohachevsky Function number 1]
18/47 in 19.6089 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Perm Function 0, d=2, beta=10]
19/47 in 26.5208 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Rotated HYper-Ellipsoid Function]
20/47 in  3.4785 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sphere Function]
21/47 in  4.8014 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Sum of Different Powers Function]
22/47 in  5.2505 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Sum Squares Function]
23/47 in  8.0968 seconds:   -49.994434, supposed  -200.000000 =>   150.005566 difference [Trid Function]
24/47 in  5.0946 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Booth Function]
25/47 in  5.0856 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Matyas Function]
26/47 in  7.1521 seconds:  -724.479527, supposed    -1.913300 =>   722.566227 difference [McCormick Function]
27/47 in 16.5868 seconds:     0.000437, supposed     0.000000 =>     0.000437 difference [Power Sum Function]
28/47 in  6.4025 seconds:     0.000000, supposed     0.000000 =>     0.000000 difference [Zakharov Function]
29/47 in  6.4566 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Three-Hump Camel Function]
30/47 in  7.7280 seconds:    -1.031628, supposed    -1.031600 =>     0.000028 difference [Sixth-Hump Camel Function]
31/47 in  8.2800 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Dixon-Price Function]
32/47 in  7.0610 seconds:     0.000003, supposed     0.000000 =>     0.000003 difference [Rosenbrock Function]
33/47 in 22.1177 seconds:    23.809434, supposed     0.998000 =>    22.811434 difference [De Jong Function number 5]
34/47 in  6.0223 seconds:    -0.997678, supposed    -1.000000 =>     0.002322 difference [Easom Function]
35/47 in  6.6287 seconds:    -1.997497, supposed    -1.801300 =>     0.196197 difference [Michalewicz Function]
36/47 in  7.2031 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Beale Function]
37/47 in  8.5434 seconds:     0.397888, supposed     0.397887 =>     0.000001 difference [Branin Function]
38/47 in 10.9881 seconds:     0.002068, supposed     0.000000 =>     0.002068 difference [Colville Function]
39/47 in  6.1574 seconds:  -510.883737, supposed    -6.020740 =>   504.862997 difference [Forrester et al. Function]
40/47 in 10.9638 seconds:     3.000010, supposed     3.000000 =>     0.000010 difference [Goldstein-Price Function]
41/47 in 24.1832 seconds:    -2.927386, supposed    -3.862780 =>     0.935394 difference [Hartmann 3-Dimensional Function]
42/47 in 26.4498 seconds:    -1.091476, supposed    -3.135474 =>     2.043998 difference [Hartmann 4-Dimensional Function]
43/47 in 27.2223 seconds:    -1.592905, supposed    -3.223700 =>     1.630795 difference [Hartmann 6-Dimensional Function]
44/47 in 20.3054 seconds:     0.000001, supposed     0.000000 =>     0.000001 difference [Perm Function d=2, beta=0.5]
45/47 in 104.7935 seconds:     0.179925, supposed     0.000000 =>     0.179925 difference [Powell Function]
46/47 in 29.6162 seconds:    -0.849542, supposed   -10.536400 =>     9.686858 difference [Shekel Function]
47/47 in  4.7544 seconds:   -64.195612, supposed   -78.331980 =>    14.136368 difference [Styblinski-Tang Function]

12.4979s +-15.6021s average computation time.
Average difference: 67.8377 +- 212.1686.
0 fatal errors.
27 errors.
20 perfect results.