Note
Click here to download the full example code
From Serial to Parallel Evaluations#
Author(s): Romain Egele.
This example demonstrates the advantages of parallel evaluations over serial evaluations. We start by defining an artificial black-box run-function by using the Ackley function:
We will use the time.sleep function to simulate a budget of 2 secondes of execution in average which helps illustrate the advantage of parallel evaluations. The @profile decorator is useful to collect starting/ending time of the run-function execution which help us know exactly when we are inside the black-box. When using this decorator, the run-function will return a dictionnary with 2 new keys "timestamp_start" and "timestamp_end". The run-function is defined in a separate module because of the “multiprocessing” backend that we are using in this example.
"""Set of Black-Box functions useful to build examples.
"""
import time
import numpy as np
from deephyper.evaluator import profile
def ackley(x, a=20, b=0.2, c=2 * np.pi):
d = len(x)
s1 = np.sum(x**2)
s2 = np.sum(np.cos(c * x))
term1 = -a * np.exp(-b * np.sqrt(s1 / d))
term2 = -np.exp(s2 / d)
y = term1 + term2 + a + np.exp(1)
return y
@profile
def run_ackley(config, sleep_loc=2, sleep_scale=0.5):
# to simulate the computation of an expensive black-box
if sleep_loc > 0:
t_sleep = np.random.normal(loc=sleep_loc, scale=sleep_scale)
t_sleep = max(t_sleep, 0)
time.sleep(t_sleep)
x = np.array([config[k] for k in config if "x" in k])
x = np.asarray_chkfinite(x) # ValueError if any NaN or Inf
return -ackley(x) # maximisation is performed
After defining the black-box we can continue with the definition of our main script:
import black_box_util as black_box
Then we define the variable(s) we want to optimize. For this problem we optimize Ackley in a 2-dimensional search space, the true minimul is located at (0, 0).
Out:
Configuration space object:
Hyperparameters:
x0, Type: UniformFloat, Range: [-32.768, 32.768], Default: 0.0
x1, Type: UniformFloat, Range: [-32.768, 32.768], Default: 0.0
Then we define serial search by creation a "serial"-evaluator and we execute the search with a fixed time-budget of 2 minutes (i.e., 120 secondes).
if __name__ == "__main__":
from deephyper.evaluator import Evaluator
from deephyper.evaluator.callback import TqdmCallback
from deephyper.search.hps import CBO
# we give a budget of 2 minutes for each search
timeout = 120
serial_evaluator = Evaluator.create(
black_box.run_ackley,
method="serial",
method_kwargs={"callbacks": [TqdmCallback()]},
)
results = {}
serial_search = CBO(problem, serial_evaluator, random_state=42)
results["serial"] = serial_search.search(timeout=timeout)
Out:
0it [00:00, ?it/s]
1it [00:00, 3269.14it/s, objective=-21.2]
2it [00:02, 1.17s/it, objective=-21.2]
2it [00:02, 1.17s/it, objective=-19.9]
3it [00:04, 1.52s/it, objective=-19.9]
3it [00:04, 1.52s/it, objective=-19.9]
4it [00:06, 1.90s/it, objective=-19.9]
4it [00:06, 1.90s/it, objective=-19.9]
5it [00:09, 2.06s/it, objective=-19.9]
5it [00:09, 2.06s/it, objective=-19.9]
6it [00:10, 1.92s/it, objective=-19.9]
6it [00:10, 1.92s/it, objective=-19.9]
7it [00:12, 1.84s/it, objective=-19.9]
7it [00:12, 1.84s/it, objective=-19.9]
8it [00:14, 1.85s/it, objective=-19.9]
8it [00:14, 1.85s/it, objective=-19.9]
9it [00:16, 2.04s/it, objective=-19.9]
9it [00:16, 2.04s/it, objective=-19.9]
10it [00:19, 2.12s/it, objective=-19.9]
10it [00:19, 2.12s/it, objective=-11.3]
11it [00:21, 2.17s/it, objective=-11.3]
11it [00:21, 2.17s/it, objective=-9.86]
12it [00:23, 2.20s/it, objective=-9.86]
12it [00:23, 2.20s/it, objective=-9.86]
13it [00:24, 1.79s/it, objective=-9.86]
13it [00:24, 1.79s/it, objective=-9.86]
14it [00:26, 1.71s/it, objective=-9.86]
14it [00:26, 1.71s/it, objective=-9.86]
15it [00:28, 2.00s/it, objective=-9.86]
15it [00:28, 2.00s/it, objective=-9.86]
16it [00:30, 1.90s/it, objective=-9.86]
16it [00:30, 1.90s/it, objective=-3.64]
17it [00:33, 2.16s/it, objective=-3.64]
17it [00:33, 2.16s/it, objective=-2.66]
18it [00:36, 2.46s/it, objective=-2.66]
18it [00:36, 2.46s/it, objective=-2.66]
19it [00:38, 2.46s/it, objective=-2.66]
19it [00:38, 2.46s/it, objective=-2.66]
20it [00:40, 2.36s/it, objective=-2.66]
20it [00:40, 2.36s/it, objective=-2.66]
21it [00:43, 2.40s/it, objective=-2.66]
21it [00:43, 2.40s/it, objective=-2.66]
22it [00:46, 2.44s/it, objective=-2.66]
22it [00:46, 2.44s/it, objective=-2.66]
23it [00:47, 2.22s/it, objective=-2.66]
23it [00:47, 2.22s/it, objective=-2.66]
24it [00:49, 2.19s/it, objective=-2.66]
24it [00:49, 2.19s/it, objective=-2.66]
25it [00:51, 2.07s/it, objective=-2.66]
25it [00:51, 2.07s/it, objective=-2.66]
26it [00:53, 2.12s/it, objective=-2.66]
26it [00:53, 2.12s/it, objective=-2.66]
27it [00:56, 2.19s/it, objective=-2.66]
27it [00:56, 2.19s/it, objective=-2.66]
28it [00:59, 2.56s/it, objective=-2.66]
28it [00:59, 2.56s/it, objective=-2.66]
29it [01:00, 2.13s/it, objective=-2.66]
29it [01:00, 2.13s/it, objective=-2.66]
30it [01:02, 2.12s/it, objective=-2.66]
30it [01:02, 2.12s/it, objective=-2.66]
31it [01:04, 2.10s/it, objective=-2.66]
31it [01:04, 2.10s/it, objective=-2.66]
32it [01:07, 2.16s/it, objective=-2.66]
32it [01:07, 2.16s/it, objective=-2.66]
33it [01:08, 1.89s/it, objective=-2.66]
33it [01:08, 1.89s/it, objective=-2.66]
34it [01:10, 1.99s/it, objective=-2.66]
34it [01:10, 1.99s/it, objective=-2.66]
35it [01:12, 2.00s/it, objective=-2.66]
35it [01:12, 2.00s/it, objective=-2.66]
36it [01:15, 2.25s/it, objective=-2.66]
36it [01:15, 2.25s/it, objective=-2.66]
37it [01:17, 2.27s/it, objective=-2.66]
37it [01:17, 2.27s/it, objective=-2.66]
38it [01:19, 2.17s/it, objective=-2.66]
38it [01:19, 2.17s/it, objective=-2.66]
39it [01:22, 2.31s/it, objective=-2.66]
39it [01:22, 2.31s/it, objective=-2.66]
40it [01:24, 2.27s/it, objective=-2.66]
40it [01:24, 2.27s/it, objective=-2.66]
41it [01:27, 2.35s/it, objective=-2.66]
41it [01:27, 2.35s/it, objective=-2.66]
42it [01:29, 2.43s/it, objective=-2.66]
42it [01:29, 2.43s/it, objective=-2.66]
43it [01:31, 2.15s/it, objective=-2.66]
43it [01:31, 2.15s/it, objective=-2.66]
44it [01:33, 2.28s/it, objective=-2.66]
44it [01:33, 2.28s/it, objective=-2.66]
45it [01:36, 2.30s/it, objective=-2.66]
45it [01:36, 2.30s/it, objective=-2.66]
46it [01:38, 2.41s/it, objective=-2.66]
46it [01:38, 2.41s/it, objective=-2.66]
47it [01:40, 2.27s/it, objective=-2.66]
47it [01:40, 2.27s/it, objective=-2.66]
48it [01:43, 2.31s/it, objective=-2.66]
48it [01:43, 2.31s/it, objective=-2.66]
49it [01:45, 2.25s/it, objective=-2.66]
49it [01:45, 2.25s/it, objective=-2.66]
50it [01:48, 2.58s/it, objective=-2.66]
50it [01:48, 2.58s/it, objective=-2.66]
51it [01:50, 2.42s/it, objective=-2.66]
51it [01:50, 2.42s/it, objective=-2.66]
52it [01:52, 2.17s/it, objective=-2.66]
52it [01:52, 2.17s/it, objective=-2.66]
53it [01:55, 2.34s/it, objective=-2.66]
53it [01:55, 2.34s/it, objective=-2.66]
54it [01:57, 2.49s/it, objective=-2.66]
54it [01:57, 2.49s/it, objective=-2.66]
After, executing the serial-search for 2 minutes we can create a parallel search which uses the "process"-evaluator and defines 5 parallel workers. The search is also executed for 2 minutes.
if __name__ == "__main__":
parallel_evaluator = Evaluator.create(
black_box.run_ackley,
method="process",
method_kwargs={"num_workers": 5, "callbacks": [TqdmCallback()]},
)
parallel_search = CBO(problem, parallel_evaluator, random_state=42)
results["parallel"] = parallel_search.search(timeout=timeout)
Out:
0it [00:00, ?it/s]
1it [00:00, 6990.51it/s, objective=-21.5]
2it [00:00, 18.17it/s, objective=-21.5]
2it [00:00, 18.17it/s, objective=-21.2]
3it [00:00, 18.17it/s, objective=-21.2]
4it [00:00, 6.14it/s, objective=-21.2]
4it [00:00, 6.14it/s, objective=-20.6]
5it [00:01, 3.00it/s, objective=-20.6]
5it [00:01, 3.00it/s, objective=-20.6]
6it [00:02, 1.59it/s, objective=-20.6]
6it [00:02, 1.59it/s, objective=-20.6]
7it [00:02, 1.91it/s, objective=-20.6]
7it [00:02, 1.91it/s, objective=-20.6]
8it [00:03, 1.91it/s, objective=-19.9]
9it [00:03, 2.58it/s, objective=-19.9]
9it [00:03, 2.58it/s, objective=-19.9]
10it [00:04, 1.94it/s, objective=-19.9]
10it [00:04, 1.94it/s, objective=-19.9]
11it [00:04, 1.82it/s, objective=-19.9]
11it [00:04, 1.82it/s, objective=-19.9]
12it [00:05, 1.88it/s, objective=-19.9]
12it [00:05, 1.88it/s, objective=-19.9]
13it [00:05, 1.91it/s, objective=-19.9]
13it [00:05, 1.91it/s, objective=-11.3]
14it [00:06, 2.24it/s, objective=-11.3]
14it [00:06, 2.24it/s, objective=-11.3]
15it [00:07, 1.64it/s, objective=-11.3]
15it [00:07, 1.64it/s, objective=-11.3]
16it [00:07, 1.98it/s, objective=-11.3]
16it [00:07, 1.98it/s, objective=-11.3]
17it [00:08, 1.56it/s, objective=-11.3]
17it [00:08, 1.56it/s, objective=-11.3]
18it [00:08, 1.82it/s, objective=-11.3]
18it [00:08, 1.82it/s, objective=-10.8]
19it [00:09, 2.15it/s, objective=-10.8]
19it [00:09, 2.15it/s, objective=-10.8]
20it [00:09, 1.73it/s, objective=-10.8]
20it [00:09, 1.73it/s, objective=-10.7]
21it [00:10, 1.35it/s, objective=-10.7]
21it [00:10, 1.35it/s, objective=-10.5]
22it [00:11, 1.67it/s, objective=-10.5]
22it [00:11, 1.67it/s, objective=-10.5]
23it [00:11, 1.99it/s, objective=-10.5]
23it [00:11, 1.99it/s, objective=-6.17]
24it [00:12, 1.62it/s, objective=-6.17]
24it [00:12, 1.62it/s, objective=-6.17]
25it [00:13, 1.59it/s, objective=-6.17]
25it [00:13, 1.59it/s, objective=-6.17]
26it [00:13, 1.47it/s, objective=-6.17]
26it [00:13, 1.47it/s, objective=-6.17]
27it [00:14, 1.61it/s, objective=-6.17]
27it [00:14, 1.61it/s, objective=-6.17]
28it [00:14, 1.84it/s, objective=-6.17]
28it [00:14, 1.84it/s, objective=-6.17]
29it [00:15, 1.50it/s, objective=-6.17]
29it [00:15, 1.50it/s, objective=-5.19]
30it [00:16, 1.39it/s, objective=-5.19]
30it [00:16, 1.39it/s, objective=-5.19]
31it [00:16, 1.63it/s, objective=-5.19]
31it [00:16, 1.63it/s, objective=-5.19]
32it [00:17, 1.93it/s, objective=-5.19]
32it [00:17, 1.93it/s, objective=-5.19]
33it [00:18, 1.31it/s, objective=-5.19]
33it [00:18, 1.31it/s, objective=-5.19]
34it [00:18, 1.54it/s, objective=-5.19]
34it [00:18, 1.54it/s, objective=-5.19]
35it [00:19, 1.47it/s, objective=-5.19]
35it [00:19, 1.47it/s, objective=-5.19]
36it [00:19, 1.77it/s, objective=-5.19]
36it [00:19, 1.77it/s, objective=-5.19]
37it [00:20, 1.98it/s, objective=-5.19]
37it [00:20, 1.98it/s, objective=-5.19]
38it [00:21, 1.27it/s, objective=-5.19]
38it [00:21, 1.27it/s, objective=-5.19]
39it [00:22, 1.54it/s, objective=-5.19]
39it [00:22, 1.54it/s, objective=-5.19]
40it [00:22, 1.38it/s, objective=-5.19]
40it [00:22, 1.38it/s, objective=-5.19]
41it [00:23, 1.61it/s, objective=-5.19]
41it [00:23, 1.61it/s, objective=-5.19]
42it [00:23, 1.86it/s, objective=-5.19]
42it [00:23, 1.86it/s, objective=-5.19]
43it [00:24, 1.60it/s, objective=-5.19]
43it [00:24, 1.60it/s, objective=-5.19]
44it [00:24, 1.82it/s, objective=-5.19]
44it [00:24, 1.82it/s, objective=-5.19]
45it [00:25, 1.41it/s, objective=-5.19]
45it [00:25, 1.41it/s, objective=-5.19]
46it [00:26, 1.42it/s, objective=-5.19]
46it [00:26, 1.42it/s, objective=-5.19]
47it [00:27, 1.65it/s, objective=-5.19]
47it [00:27, 1.65it/s, objective=-5.19]
48it [00:27, 1.56it/s, objective=-5.19]
48it [00:27, 1.56it/s, objective=-4.97]
49it [00:28, 1.85it/s, objective=-4.97]
49it [00:28, 1.85it/s, objective=-3.23]
50it [00:28, 2.03it/s, objective=-3.23]
50it [00:28, 2.03it/s, objective=-3.23]
51it [00:29, 1.36it/s, objective=-3.23]
51it [00:29, 1.36it/s, objective=-2.7]
52it [00:30, 1.63it/s, objective=-2.7]
52it [00:30, 1.63it/s, objective=-2.7]
53it [00:30, 1.53it/s, objective=-2.7]
53it [00:30, 1.53it/s, objective=-2.7]
54it [00:31, 1.75it/s, objective=-2.7]
54it [00:31, 1.75it/s, objective=-2.69]
55it [00:31, 1.75it/s, objective=-2.01]
56it [00:32, 1.73it/s, objective=-2.01]
56it [00:32, 1.73it/s, objective=-2.01]
57it [00:32, 1.82it/s, objective=-2.01]
57it [00:32, 1.82it/s, objective=-2.01]
58it [00:33, 1.79it/s, objective=-2.01]
58it [00:33, 1.79it/s, objective=-2.01]
59it [00:33, 1.88it/s, objective=-2.01]
59it [00:33, 1.88it/s, objective=-2.01]
60it [00:34, 1.77it/s, objective=-2.01]
60it [00:34, 1.77it/s, objective=-2.01]
61it [00:34, 1.95it/s, objective=-2.01]
61it [00:34, 1.95it/s, objective=-2.01]
62it [00:35, 2.21it/s, objective=-2.01]
62it [00:35, 2.21it/s, objective=-2.01]
63it [00:35, 2.45it/s, objective=-2.01]
63it [00:35, 2.45it/s, objective=-2.01]
64it [00:36, 1.47it/s, objective=-2.01]
64it [00:36, 1.47it/s, objective=-2.01]
65it [00:37, 1.46it/s, objective=-2.01]
65it [00:37, 1.46it/s, objective=-2.01]
66it [00:37, 1.75it/s, objective=-2.01]
66it [00:37, 1.75it/s, objective=-2.01]
67it [00:38, 1.94it/s, objective=-2.01]
67it [00:38, 1.94it/s, objective=-2.01]
68it [00:38, 2.06it/s, objective=-2.01]
68it [00:38, 2.06it/s, objective=-2.01]
69it [00:39, 1.91it/s, objective=-2.01]
69it [00:39, 1.91it/s, objective=-2.01]
70it [00:40, 1.19it/s, objective=-2.01]
70it [00:40, 1.19it/s, objective=-2.01]
71it [00:41, 1.46it/s, objective=-2.01]
71it [00:41, 1.46it/s, objective=-2.01]
72it [00:41, 1.75it/s, objective=-2.01]
72it [00:41, 1.75it/s, objective=-2.01]
73it [00:41, 1.89it/s, objective=-2.01]
73it [00:41, 1.89it/s, objective=-2.01]
74it [00:42, 2.04it/s, objective=-2.01]
74it [00:42, 2.04it/s, objective=-2.01]
75it [00:44, 1.02it/s, objective=-2.01]
75it [00:44, 1.02it/s, objective=-2.01]
76it [00:44, 1.27it/s, objective=-2.01]
76it [00:44, 1.27it/s, objective=-2.01]
77it [00:45, 1.49it/s, objective=-2.01]
77it [00:45, 1.49it/s, objective=-2.01]
78it [00:45, 1.49it/s, objective=-2.01]
79it [00:45, 1.49it/s, objective=-2.01]
80it [00:47, 1.42it/s, objective=-2.01]
80it [00:47, 1.42it/s, objective=-2.01]
81it [00:47, 1.50it/s, objective=-2.01]
81it [00:47, 1.50it/s, objective=-2.01]
82it [00:48, 1.72it/s, objective=-2.01]
82it [00:48, 1.72it/s, objective=-2.01]
83it [00:48, 1.70it/s, objective=-2.01]
83it [00:48, 1.70it/s, objective=-2.01]
84it [00:49, 1.87it/s, objective=-2.01]
84it [00:49, 1.87it/s, objective=-2.01]
85it [00:50, 1.47it/s, objective=-2.01]
85it [00:50, 1.47it/s, objective=-2.01]
86it [00:51, 1.36it/s, objective=-2.01]
86it [00:51, 1.36it/s, objective=-2.01]
87it [00:51, 1.58it/s, objective=-2.01]
87it [00:51, 1.58it/s, objective=-2.01]
88it [00:51, 1.70it/s, objective=-2.01]
88it [00:51, 1.70it/s, objective=-2.01]
89it [00:52, 1.96it/s, objective=-2.01]
89it [00:52, 1.96it/s, objective=-2.01]
90it [00:52, 2.21it/s, objective=-2.01]
90it [00:52, 2.21it/s, objective=-2.01]
91it [00:54, 1.31it/s, objective=-2.01]
91it [00:54, 1.31it/s, objective=-2.01]
92it [00:54, 1.48it/s, objective=-2.01]
92it [00:54, 1.48it/s, objective=-2.01]
93it [00:54, 1.77it/s, objective=-2.01]
93it [00:54, 1.77it/s, objective=-2.01]
94it [00:55, 1.94it/s, objective=-2.01]
94it [00:55, 1.94it/s, objective=-2.01]
95it [00:56, 1.66it/s, objective=-2.01]
95it [00:56, 1.66it/s, objective=-2.01]
96it [00:57, 1.39it/s, objective=-2.01]
96it [00:57, 1.39it/s, objective=-2.01]
97it [00:57, 1.54it/s, objective=-2.01]
97it [00:57, 1.54it/s, objective=-2.01]
98it [00:57, 1.74it/s, objective=-2.01]
98it [00:57, 1.74it/s, objective=-2.01]
99it [00:58, 2.02it/s, objective=-2.01]
99it [00:58, 2.02it/s, objective=-2.01]
100it [00:59, 1.50it/s, objective=-2.01]
100it [00:59, 1.50it/s, objective=-2.01]
101it [00:59, 1.70it/s, objective=-2.01]
101it [00:59, 1.70it/s, objective=-2.01]
102it [01:00, 1.63it/s, objective=-2.01]
102it [01:00, 1.63it/s, objective=-2.01]
103it [01:00, 1.91it/s, objective=-2.01]
103it [01:00, 1.91it/s, objective=-2.01]
104it [01:01, 1.56it/s, objective=-2.01]
104it [01:01, 1.56it/s, objective=-2.01]
105it [01:01, 1.83it/s, objective=-2.01]
105it [01:01, 1.83it/s, objective=-2.01]
106it [01:03, 1.23it/s, objective=-2.01]
106it [01:03, 1.23it/s, objective=-2.01]
107it [01:03, 1.39it/s, objective=-2.01]
107it [01:03, 1.39it/s, objective=-2.01]
108it [01:04, 1.55it/s, objective=-2.01]
108it [01:04, 1.55it/s, objective=-.378]
109it [01:04, 1.83it/s, objective=-.378]
109it [01:04, 1.83it/s, objective=-.378]
110it [01:04, 1.83it/s, objective=-.378]
111it [01:06, 1.47it/s, objective=-.378]
111it [01:06, 1.47it/s, objective=-.378]
112it [01:07, 1.37it/s, objective=-.378]
112it [01:07, 1.37it/s, objective=-.378]
113it [01:07, 1.60it/s, objective=-.378]
113it [01:07, 1.60it/s, objective=-.378]
114it [01:07, 1.78it/s, objective=-.378]
114it [01:07, 1.78it/s, objective=-.0826]
115it [01:08, 2.02it/s, objective=-.0826]
115it [01:08, 2.02it/s, objective=-.0826]
116it [01:09, 1.33it/s, objective=-.0826]
116it [01:09, 1.33it/s, objective=-.0826]
117it [01:10, 1.46it/s, objective=-.0826]
117it [01:10, 1.46it/s, objective=-.0826]
118it [01:10, 1.74it/s, objective=-.0826]
118it [01:10, 1.74it/s, objective=-.0826]
119it [01:10, 2.00it/s, objective=-.0826]
119it [01:10, 2.00it/s, objective=-.0826]
120it [01:10, 2.00it/s, objective=-.0826]
121it [01:12, 1.41it/s, objective=-.0826]
121it [01:12, 1.41it/s, objective=-.0826]
122it [01:13, 1.61it/s, objective=-.0826]
122it [01:13, 1.61it/s, objective=-.0826]
123it [01:14, 1.31it/s, objective=-.0826]
123it [01:14, 1.31it/s, objective=-.0826]
124it [01:14, 1.51it/s, objective=-.0826]
124it [01:14, 1.51it/s, objective=-.0826]
125it [01:14, 1.51it/s, objective=-.0826]
126it [01:15, 1.51it/s, objective=-.0826]
126it [01:15, 1.51it/s, objective=-.0826]
127it [01:16, 1.72it/s, objective=-.0826]
127it [01:16, 1.72it/s, objective=-.0826]
128it [01:17, 1.53it/s, objective=-.0826]
128it [01:17, 1.53it/s, objective=-.0826]
129it [01:17, 1.44it/s, objective=-.0826]
129it [01:17, 1.44it/s, objective=-.0826]
130it [01:18, 1.70it/s, objective=-.0826]
130it [01:18, 1.70it/s, objective=-.0826]
131it [01:19, 1.42it/s, objective=-.0826]
131it [01:19, 1.42it/s, objective=-.0826]
132it [01:19, 1.57it/s, objective=-.0826]
132it [01:19, 1.57it/s, objective=-.0826]
133it [01:20, 1.74it/s, objective=-.0826]
133it [01:20, 1.74it/s, objective=-.0826]
134it [01:20, 1.91it/s, objective=-.0826]
134it [01:20, 1.91it/s, objective=-.0826]
135it [01:20, 2.17it/s, objective=-.0826]
135it [01:20, 2.17it/s, objective=-.0826]
136it [01:22, 1.31it/s, objective=-.0826]
136it [01:22, 1.31it/s, objective=-.0826]
137it [01:22, 1.57it/s, objective=-.0826]
137it [01:22, 1.57it/s, objective=-.0826]
138it [01:23, 1.77it/s, objective=-.0826]
138it [01:23, 1.77it/s, objective=-.0826]
139it [01:23, 1.72it/s, objective=-.0826]
139it [01:23, 1.72it/s, objective=-.0826]
140it [01:24, 1.72it/s, objective=-.0826]
140it [01:24, 1.72it/s, objective=-.0826]
141it [01:24, 1.65it/s, objective=-.0826]
141it [01:24, 1.65it/s, objective=-.0826]
142it [01:25, 1.66it/s, objective=-.0826]
142it [01:25, 1.66it/s, objective=-.0826]
143it [01:26, 1.60it/s, objective=-.0826]
143it [01:26, 1.60it/s, objective=-.0826]
144it [01:26, 1.80it/s, objective=-.0826]
144it [01:26, 1.80it/s, objective=-.0826]
145it [01:27, 1.71it/s, objective=-.0826]
145it [01:27, 1.71it/s, objective=-.0826]
146it [01:27, 1.98it/s, objective=-.0826]
146it [01:27, 1.98it/s, objective=-.0826]
147it [01:28, 1.82it/s, objective=-.0826]
147it [01:28, 1.82it/s, objective=-.0826]
148it [01:29, 1.41it/s, objective=-.0826]
148it [01:29, 1.41it/s, objective=-.0826]
149it [01:29, 1.70it/s, objective=-.0826]
149it [01:29, 1.70it/s, objective=-.0826]
150it [01:30, 1.69it/s, objective=-.0826]
150it [01:30, 1.69it/s, objective=-.0826]
151it [01:30, 1.71it/s, objective=-.0826]
151it [01:30, 1.71it/s, objective=-.0826]
152it [01:31, 1.95it/s, objective=-.0826]
152it [01:31, 1.95it/s, objective=-.0826]
153it [01:33, 1.09it/s, objective=-.0826]
153it [01:33, 1.09it/s, objective=-.0826]
154it [01:33, 1.30it/s, objective=-.0826]
154it [01:33, 1.30it/s, objective=-.0826]
155it [01:33, 1.30it/s, objective=-.0826]
156it [01:33, 1.30it/s, objective=-.0826]
157it [01:34, 2.19it/s, objective=-.0826]
157it [01:34, 2.19it/s, objective=-.0826]
158it [01:36, 1.28it/s, objective=-.0826]
158it [01:36, 1.28it/s, objective=-.0826]
159it [01:36, 1.48it/s, objective=-.0826]
159it [01:36, 1.48it/s, objective=-.0826]
160it [01:36, 1.61it/s, objective=-.0826]
160it [01:36, 1.61it/s, objective=-.0826]
161it [01:37, 1.77it/s, objective=-.0826]
161it [01:37, 1.77it/s, objective=-.0826]
162it [01:37, 2.01it/s, objective=-.0826]
162it [01:37, 2.01it/s, objective=-.0826]
163it [01:38, 1.57it/s, objective=-.0826]
163it [01:38, 1.57it/s, objective=-.0826]
164it [01:39, 1.27it/s, objective=-.0826]
164it [01:39, 1.27it/s, objective=-.0826]
165it [01:40, 1.47it/s, objective=-.0826]
165it [01:40, 1.47it/s, objective=-.0826]
166it [01:40, 1.72it/s, objective=-.0826]
166it [01:40, 1.72it/s, objective=-.0826]
167it [01:40, 1.90it/s, objective=-.0826]
167it [01:40, 1.90it/s, objective=-.0826]
168it [01:41, 2.04it/s, objective=-.0826]
168it [01:41, 2.04it/s, objective=-.0826]
169it [01:43, 1.02s/it, objective=-.0826]
169it [01:43, 1.02s/it, objective=-.0826]
170it [01:43, 1.22it/s, objective=-.0826]
170it [01:43, 1.22it/s, objective=-.0826]
171it [01:43, 1.22it/s, objective=-.0826]
172it [01:44, 1.75it/s, objective=-.0826]
172it [01:44, 1.75it/s, objective=-.0826]
173it [01:44, 1.96it/s, objective=-.0826]
173it [01:44, 1.96it/s, objective=-.0826]
174it [01:45, 1.54it/s, objective=-.0826]
174it [01:45, 1.54it/s, objective=-.0826]
175it [01:46, 1.30it/s, objective=-.0826]
175it [01:46, 1.30it/s, objective=-.0826]
176it [01:47, 1.56it/s, objective=-.0826]
176it [01:47, 1.56it/s, objective=-.0826]
177it [01:47, 1.70it/s, objective=-.0826]
177it [01:47, 1.70it/s, objective=-.0826]
178it [01:48, 1.74it/s, objective=-.0826]
178it [01:48, 1.74it/s, objective=-.0826]
179it [01:48, 2.00it/s, objective=-.0826]
179it [01:48, 2.00it/s, objective=-.0826]
180it [01:49, 1.75it/s, objective=-.0826]
180it [01:49, 1.75it/s, objective=-.0826]
181it [01:50, 1.55it/s, objective=-.0826]
181it [01:50, 1.55it/s, objective=-.0826]
182it [01:50, 1.53it/s, objective=-.0826]
182it [01:50, 1.53it/s, objective=-.0826]
183it [01:51, 1.60it/s, objective=-.0826]
183it [01:51, 1.60it/s, objective=-.0826]
184it [01:52, 1.58it/s, objective=-.0826]
184it [01:52, 1.58it/s, objective=-.0826]
185it [01:52, 1.43it/s, objective=-.0826]
185it [01:52, 1.43it/s, objective=-.0826]
186it [01:53, 1.66it/s, objective=-.0826]
186it [01:53, 1.66it/s, objective=-.0826]
187it [01:53, 1.66it/s, objective=-.0826]
187it [01:53, 1.66it/s, objective=-.0826]
188it [01:54, 1.84it/s, objective=-.0826]
188it [01:54, 1.84it/s, objective=-.0826]
189it [01:55, 1.23it/s, objective=-.0826]
189it [01:55, 1.23it/s, objective=-.0826]
190it [01:56, 1.48it/s, objective=-.0826]
190it [01:56, 1.48it/s, objective=-.0826]
191it [01:56, 1.48it/s, objective=-.0826]
192it [01:57, 1.65it/s, objective=-.0826]
192it [01:57, 1.65it/s, objective=-.0826]
193it [01:57, 1.80it/s, objective=-.0826]
193it [01:57, 1.80it/s, objective=-.0826]
Finally, we plot the results from the collected DataFrame. The execution time is used as the x-axis which help-us vizualise the advantages of the parallel search.
if __name__ == "__main__":
import matplotlib.pyplot as plt
plt.figure()
for strategy, df in results.items():
plt.scatter(df.timestamp_end, df.objective, label=strategy)
plt.plot(df.timestamp_end, df.objective.cummax())
plt.xlabel("Time (sec.)")
plt.ylabel("Objective")
plt.grid()
plt.legend()
plt.show()
Total running time of the script: ( 4 minutes 1.933 seconds)