Note
Go to the end to download the full example code.
Applying Transfer Learning to Hyperparameter Optimization#
Author(s): Romain Egele.
In this example, we demonstrate how to leverage transfer learning for hyperparameter optimization. Imagine you are working on multiple related tasks, such as optimizing the hyperparameters of neural networks for various datasets. It’s reasonable to expect that similar hyperparameter configurations might perform well across these datasets, even if some minor adjustments are needed to fine-tune performance.
By conducting a thorough (and potentially expensive) search on one task, you can reuse the resulting hyperparameter set to guide and accelerate optimization for subsequent tasks. This approach reduces computational costs while maintaining high performance.
To illustrate, we will use a simple and computationally inexpensive example: minimizing the function \(f(x) = \sum_{i=0}^ {n-1}\). Here, the difficulty of the problem is defined by the number of variables \(n\). We’ll start by optimizing the small problem where \(n=1\). Then, we’ll apply transfer learning to optimize a larger problem where \(n=3\), comparing the results with and without transfer learning to highlight the benefits.
Let’s begin by defining the run-functions for both the small-scale and large-scale problems:
import functools
import matplotlib.pyplot as plt
from deephyper.analysis import figure_size
from deephyper.analysis.hpo import plot_search_trajectory_single_objective_hpo
from deephyper.evaluator import Evaluator
from deephyper.evaluator.callback import TqdmCallback
from deephyper.hpo import CBO, HpProblem
def run(config: dict, N: int) -> float:
# Definition of the function to minimize
y = sum([config[f"x{i}"] ** 2 for i in range(N)])
return -y # Use the `-` sign to perform minimization
n_small = 1
n_large = 3
run_small = functools.partial(run, N=n_small)
run_large = functools.partial(run, N=n_large)
Then, we can define the hyperparameter problem space based on \(n\)
N = n_small
problem_small = HpProblem()
for i in range(N):
problem_small.add_hyperparameter((-10.0, 10.0), f"x{i}")
problem_small
Configuration space object:
Hyperparameters:
x0, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0
N = n_large
problem_large = HpProblem()
for i in range(N):
problem_large.add_hyperparameter((-10.0, 10.0), f"x{i}")
problem_large
Configuration space object:
Hyperparameters:
x0, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0
x1, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0
x2, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0
Then, we define setup the search and execute it:
results = {}
max_evals = 100
evaluator_small = Evaluator.create(
run_small, method="thread", method_kwargs={"callbacks": [TqdmCallback()]}
)
search_small = CBO(problem_small, evaluator_small, random_state=42)
results_small = search_small.search(max_evals)
0%| | 0/100 [00:00<?, ?it/s]
1%| | 1/100 [00:00<00:00, 39945.75it/s, failures=0, objective=-3.23]
2%|▏ | 2/100 [00:00<00:00, 390.62it/s, failures=0, objective=-3.23]
3%|▎ | 3/100 [00:00<00:00, 301.99it/s, failures=0, objective=-1.22]
4%|▍ | 4/100 [00:00<00:00, 267.70it/s, failures=0, objective=-1.22]
5%|▌ | 5/100 [00:00<00:00, 251.28it/s, failures=0, objective=-1.22]
6%|▌ | 6/100 [00:00<00:00, 236.58it/s, failures=0, objective=-1.22]
7%|▋ | 7/100 [00:00<00:00, 226.83it/s, failures=0, objective=-1.22]
8%|▊ | 8/100 [00:00<00:00, 220.11it/s, failures=0, objective=-0.754]
9%|▉ | 9/100 [00:00<00:00, 217.10it/s, failures=0, objective=-0.754]
10%|█ | 10/100 [00:00<00:00, 215.33it/s, failures=0, objective=-0.754]
11%|█ | 11/100 [00:00<00:01, 86.58it/s, failures=0, objective=-0.754]
11%|█ | 11/100 [00:00<00:01, 86.58it/s, failures=0, objective=-0.754]
12%|█▏ | 12/100 [00:00<00:01, 86.58it/s, failures=0, objective=-0.754]
13%|█▎ | 13/100 [00:00<00:01, 86.58it/s, failures=0, objective=-0.26]
14%|█▍ | 14/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.0145]
15%|█▌ | 15/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.0145]
16%|█▌ | 16/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.0145]
17%|█▋ | 17/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.0145]
18%|█▊ | 18/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.0145]
19%|█▉ | 19/100 [00:00<00:00, 86.58it/s, failures=0, objective=-0.00148]
20%|██ | 20/100 [00:00<00:04, 19.86it/s, failures=0, objective=-0.00148]
20%|██ | 20/100 [00:00<00:04, 19.86it/s, failures=0, objective=-0.000458]
21%|██ | 21/100 [00:00<00:03, 19.86it/s, failures=0, objective=-0.000458]
22%|██▏ | 22/100 [00:01<00:03, 19.86it/s, failures=0, objective=-0.000458]
23%|██▎ | 23/100 [00:01<00:03, 19.86it/s, failures=0, objective=-0.000103]
24%|██▍ | 24/100 [00:01<00:03, 19.86it/s, failures=0, objective=-1.04e-6]
25%|██▌ | 25/100 [00:01<00:04, 16.62it/s, failures=0, objective=-1.04e-6]
25%|██▌ | 25/100 [00:01<00:04, 16.62it/s, failures=0, objective=-1.04e-6]
26%|██▌ | 26/100 [00:01<00:04, 16.62it/s, failures=0, objective=-1.04e-6]
27%|██▋ | 27/100 [00:01<00:04, 16.62it/s, failures=0, objective=-1.04e-6]
28%|██▊ | 28/100 [00:01<00:04, 14.77it/s, failures=0, objective=-1.04e-6]
28%|██▊ | 28/100 [00:01<00:04, 14.77it/s, failures=0, objective=-1.04e-6]
29%|██▉ | 29/100 [00:01<00:04, 14.77it/s, failures=0, objective=-5.73e-9]
30%|███ | 30/100 [00:01<00:04, 14.77it/s, failures=0, objective=-5.73e-9]
31%|███ | 31/100 [00:01<00:04, 14.02it/s, failures=0, objective=-5.73e-9]
31%|███ | 31/100 [00:01<00:04, 14.02it/s, failures=0, objective=-5.73e-9]
32%|███▏ | 32/100 [00:01<00:04, 14.02it/s, failures=0, objective=-5.73e-9]
33%|███▎ | 33/100 [00:02<00:04, 13.59it/s, failures=0, objective=-5.73e-9]
33%|███▎ | 33/100 [00:02<00:04, 13.59it/s, failures=0, objective=-5.73e-9]
34%|███▍ | 34/100 [00:02<00:04, 13.59it/s, failures=0, objective=-5.73e-9]
35%|███▌ | 35/100 [00:02<00:04, 13.22it/s, failures=0, objective=-5.73e-9]
35%|███▌ | 35/100 [00:02<00:04, 13.22it/s, failures=0, objective=-5.73e-9]
36%|███▌ | 36/100 [00:02<00:04, 13.22it/s, failures=0, objective=-5.73e-9]
37%|███▋ | 37/100 [00:02<00:04, 12.91it/s, failures=0, objective=-5.73e-9]
37%|███▋ | 37/100 [00:02<00:04, 12.91it/s, failures=0, objective=-5.73e-9]
38%|███▊ | 38/100 [00:02<00:04, 12.91it/s, failures=0, objective=-5.73e-9]
39%|███▉ | 39/100 [00:02<00:04, 12.66it/s, failures=0, objective=-5.73e-9]
39%|███▉ | 39/100 [00:02<00:04, 12.66it/s, failures=0, objective=-5.73e-9]
40%|████ | 40/100 [00:02<00:04, 12.66it/s, failures=0, objective=-5.73e-9]
41%|████ | 41/100 [00:02<00:04, 12.26it/s, failures=0, objective=-5.73e-9]
41%|████ | 41/100 [00:02<00:04, 12.26it/s, failures=0, objective=-5.73e-9]
42%|████▏ | 42/100 [00:02<00:04, 12.26it/s, failures=0, objective=-5.73e-9]
43%|████▎ | 43/100 [00:02<00:04, 12.10it/s, failures=0, objective=-5.73e-9]
43%|████▎ | 43/100 [00:02<00:04, 12.10it/s, failures=0, objective=-5.73e-9]
44%|████▍ | 44/100 [00:02<00:04, 12.10it/s, failures=0, objective=-5.73e-9]
45%|████▌ | 45/100 [00:03<00:04, 12.09it/s, failures=0, objective=-5.73e-9]
45%|████▌ | 45/100 [00:03<00:04, 12.09it/s, failures=0, objective=-5.73e-9]
46%|████▌ | 46/100 [00:03<00:04, 12.09it/s, failures=0, objective=-5.73e-9]
47%|████▋ | 47/100 [00:03<00:04, 11.98it/s, failures=0, objective=-5.73e-9]
47%|████▋ | 47/100 [00:03<00:04, 11.98it/s, failures=0, objective=-6.91e-10]
48%|████▊ | 48/100 [00:03<00:04, 11.98it/s, failures=0, objective=-6.91e-10]
49%|████▉ | 49/100 [00:03<00:04, 11.96it/s, failures=0, objective=-6.91e-10]
49%|████▉ | 49/100 [00:03<00:04, 11.96it/s, failures=0, objective=-6.91e-10]
50%|█████ | 50/100 [00:03<00:04, 11.96it/s, failures=0, objective=-6.91e-10]
51%|█████ | 51/100 [00:03<00:04, 11.37it/s, failures=0, objective=-6.91e-10]
51%|█████ | 51/100 [00:03<00:04, 11.37it/s, failures=0, objective=-6.91e-10]
52%|█████▏ | 52/100 [00:03<00:04, 11.37it/s, failures=0, objective=-6.91e-10]
53%|█████▎ | 53/100 [00:03<00:04, 11.53it/s, failures=0, objective=-6.91e-10]
53%|█████▎ | 53/100 [00:03<00:04, 11.53it/s, failures=0, objective=-6.91e-10]
54%|█████▍ | 54/100 [00:03<00:03, 11.53it/s, failures=0, objective=-6.91e-10]
55%|█████▌ | 55/100 [00:03<00:03, 11.61it/s, failures=0, objective=-6.91e-10]
55%|█████▌ | 55/100 [00:03<00:03, 11.61it/s, failures=0, objective=-6.91e-10]
56%|█████▌ | 56/100 [00:03<00:03, 11.61it/s, failures=0, objective=-6.91e-10]
57%|█████▋ | 57/100 [00:04<00:03, 11.67it/s, failures=0, objective=-6.91e-10]
57%|█████▋ | 57/100 [00:04<00:03, 11.67it/s, failures=0, objective=-6.91e-10]
58%|█████▊ | 58/100 [00:04<00:03, 11.67it/s, failures=0, objective=-6.91e-10]
59%|█████▉ | 59/100 [00:04<00:03, 11.74it/s, failures=0, objective=-6.91e-10]
59%|█████▉ | 59/100 [00:04<00:03, 11.74it/s, failures=0, objective=-6.91e-10]
60%|██████ | 60/100 [00:04<00:03, 11.74it/s, failures=0, objective=-6.91e-10]
61%|██████ | 61/100 [00:04<00:03, 11.77it/s, failures=0, objective=-6.91e-10]
61%|██████ | 61/100 [00:04<00:03, 11.77it/s, failures=0, objective=-6.91e-10]
62%|██████▏ | 62/100 [00:04<00:03, 11.77it/s, failures=0, objective=-6.91e-10]
63%|██████▎ | 63/100 [00:04<00:03, 11.81it/s, failures=0, objective=-6.91e-10]
63%|██████▎ | 63/100 [00:04<00:03, 11.81it/s, failures=0, objective=-6.91e-10]
64%|██████▍ | 64/100 [00:04<00:03, 11.81it/s, failures=0, objective=-6.91e-10]
65%|██████▌ | 65/100 [00:04<00:03, 11.23it/s, failures=0, objective=-6.91e-10]
65%|██████▌ | 65/100 [00:04<00:03, 11.23it/s, failures=0, objective=-6.91e-10]
66%|██████▌ | 66/100 [00:04<00:03, 11.23it/s, failures=0, objective=-6.91e-10]
67%|██████▋ | 67/100 [00:05<00:03, 10.01it/s, failures=0, objective=-6.91e-10]
67%|██████▋ | 67/100 [00:05<00:03, 10.01it/s, failures=0, objective=-6.91e-10]
68%|██████▊ | 68/100 [00:05<00:03, 10.01it/s, failures=0, objective=-6.91e-10]
69%|██████▉ | 69/100 [00:05<00:03, 10.25it/s, failures=0, objective=-6.91e-10]
69%|██████▉ | 69/100 [00:05<00:03, 10.25it/s, failures=0, objective=-6.91e-10]
70%|███████ | 70/100 [00:05<00:02, 10.25it/s, failures=0, objective=-6.91e-10]
71%|███████ | 71/100 [00:05<00:02, 10.16it/s, failures=0, objective=-6.91e-10]
71%|███████ | 71/100 [00:05<00:02, 10.16it/s, failures=0, objective=-6.91e-10]
72%|███████▏ | 72/100 [00:05<00:02, 10.16it/s, failures=0, objective=-6.91e-10]
73%|███████▎ | 73/100 [00:05<00:02, 9.73it/s, failures=0, objective=-6.91e-10]
73%|███████▎ | 73/100 [00:05<00:02, 9.73it/s, failures=0, objective=-6.91e-10]
74%|███████▍ | 74/100 [00:05<00:02, 9.73it/s, failures=0, objective=-6.91e-10]
75%|███████▌ | 75/100 [00:05<00:02, 10.19it/s, failures=0, objective=-6.91e-10]
75%|███████▌ | 75/100 [00:05<00:02, 10.19it/s, failures=0, objective=-6.91e-10]
76%|███████▌ | 76/100 [00:05<00:02, 10.19it/s, failures=0, objective=-6.91e-10]
77%|███████▋ | 77/100 [00:05<00:02, 10.56it/s, failures=0, objective=-6.91e-10]
77%|███████▋ | 77/100 [00:05<00:02, 10.56it/s, failures=0, objective=-6.91e-10]
78%|███████▊ | 78/100 [00:06<00:02, 10.56it/s, failures=0, objective=-6.91e-10]
79%|███████▉ | 79/100 [00:06<00:01, 10.81it/s, failures=0, objective=-6.91e-10]
79%|███████▉ | 79/100 [00:06<00:01, 10.81it/s, failures=0, objective=-6.91e-10]
80%|████████ | 80/100 [00:06<00:01, 10.81it/s, failures=0, objective=-6.91e-10]
81%|████████ | 81/100 [00:06<00:01, 11.03it/s, failures=0, objective=-6.91e-10]
81%|████████ | 81/100 [00:06<00:01, 11.03it/s, failures=0, objective=-6.91e-10]
82%|████████▏ | 82/100 [00:06<00:01, 11.03it/s, failures=0, objective=-6.91e-10]
83%|████████▎ | 83/100 [00:06<00:01, 11.09it/s, failures=0, objective=-6.91e-10]
83%|████████▎ | 83/100 [00:06<00:01, 11.09it/s, failures=0, objective=-6.91e-10]
84%|████████▍ | 84/100 [00:06<00:01, 11.09it/s, failures=0, objective=-6.91e-10]
85%|████████▌ | 85/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
85%|████████▌ | 85/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
86%|████████▌ | 86/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
87%|████████▋ | 87/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
87%|████████▋ | 87/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
88%|████████▊ | 88/100 [00:06<00:01, 11.24it/s, failures=0, objective=-6.91e-10]
89%|████████▉ | 89/100 [00:07<00:00, 11.36it/s, failures=0, objective=-6.91e-10]
89%|████████▉ | 89/100 [00:07<00:00, 11.36it/s, failures=0, objective=-6.91e-10]
90%|█████████ | 90/100 [00:07<00:00, 11.36it/s, failures=0, objective=-6.91e-10]
91%|█████████ | 91/100 [00:07<00:00, 11.37it/s, failures=0, objective=-6.91e-10]
91%|█████████ | 91/100 [00:07<00:00, 11.37it/s, failures=0, objective=-6.91e-10]
92%|█████████▏| 92/100 [00:07<00:00, 11.37it/s, failures=0, objective=-6.91e-10]
93%|█████████▎| 93/100 [00:07<00:00, 11.45it/s, failures=0, objective=-6.91e-10]
93%|█████████▎| 93/100 [00:07<00:00, 11.45it/s, failures=0, objective=-6.91e-10]
94%|█████████▍| 94/100 [00:07<00:00, 11.45it/s, failures=0, objective=-6.91e-10]
95%|█████████▌| 95/100 [00:07<00:00, 11.58it/s, failures=0, objective=-6.91e-10]
95%|█████████▌| 95/100 [00:07<00:00, 11.58it/s, failures=0, objective=-6.91e-10]
96%|█████████▌| 96/100 [00:07<00:00, 11.58it/s, failures=0, objective=-6.91e-10]
97%|█████████▋| 97/100 [00:07<00:00, 11.53it/s, failures=0, objective=-6.91e-10]
97%|█████████▋| 97/100 [00:07<00:00, 11.53it/s, failures=0, objective=-6.91e-10]
98%|█████████▊| 98/100 [00:07<00:00, 11.53it/s, failures=0, objective=-6.91e-10]
99%|█████████▉| 99/100 [00:07<00:00, 11.59it/s, failures=0, objective=-6.91e-10]
99%|█████████▉| 99/100 [00:07<00:00, 11.59it/s, failures=0, objective=-6.91e-10]
100%|██████████| 100/100 [00:07<00:00, 11.59it/s, failures=0, objective=-6.91e-10]
evaluator_large = Evaluator.create(
run_large, method="thread", method_kwargs={"callbacks": [TqdmCallback()]}
)
search_large = CBO(problem_large, evaluator_large, random_state=42)
results["Large"] = search_large.search(max_evals)
0%| | 0/100 [00:00<?, ?it/s]
1%| | 1/100 [00:00<00:00, 33288.13it/s, failures=0, objective=-50]
2%|▏ | 2/100 [00:00<00:00, 165.62it/s, failures=0, objective=-50]
3%|▎ | 3/100 [00:00<00:00, 127.71it/s, failures=0, objective=-50]
4%|▍ | 4/100 [00:00<00:00, 114.92it/s, failures=0, objective=-50]
5%|▌ | 5/100 [00:00<00:00, 108.42it/s, failures=0, objective=-10.6]
6%|▌ | 6/100 [00:00<00:00, 104.09it/s, failures=0, objective=-10.6]
7%|▋ | 7/100 [00:00<00:00, 101.55it/s, failures=0, objective=-10.6]
8%|▊ | 8/100 [00:00<00:00, 99.57it/s, failures=0, objective=-10.6]
9%|▉ | 9/100 [00:00<00:00, 98.17it/s, failures=0, objective=-10.6]
10%|█ | 10/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
10%|█ | 10/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
11%|█ | 11/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
12%|█▏ | 12/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
13%|█▎ | 13/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
14%|█▍ | 14/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
15%|█▌ | 15/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
16%|█▌ | 16/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
17%|█▋ | 17/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
18%|█▊ | 18/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
19%|█▉ | 19/100 [00:00<00:00, 97.21it/s, failures=0, objective=-10.6]
20%|██ | 20/100 [00:00<00:04, 17.56it/s, failures=0, objective=-10.6]
20%|██ | 20/100 [00:00<00:04, 17.56it/s, failures=0, objective=-3.94]
21%|██ | 21/100 [00:01<00:04, 17.56it/s, failures=0, objective=-3.94]
22%|██▏ | 22/100 [00:01<00:04, 17.56it/s, failures=0, objective=-3.94]
23%|██▎ | 23/100 [00:01<00:04, 17.56it/s, failures=0, objective=-3.94]
24%|██▍ | 24/100 [00:01<00:04, 17.56it/s, failures=0, objective=-1.07]
25%|██▌ | 25/100 [00:01<00:05, 14.67it/s, failures=0, objective=-1.07]
25%|██▌ | 25/100 [00:01<00:05, 14.67it/s, failures=0, objective=-1.07]
26%|██▌ | 26/100 [00:01<00:05, 14.67it/s, failures=0, objective=-1.07]
27%|██▋ | 27/100 [00:01<00:04, 14.67it/s, failures=0, objective=-1.07]
28%|██▊ | 28/100 [00:01<00:04, 14.67it/s, failures=0, objective=-1.07]
29%|██▉ | 29/100 [00:01<00:05, 13.17it/s, failures=0, objective=-1.07]
29%|██▉ | 29/100 [00:01<00:05, 13.17it/s, failures=0, objective=-1.07]
30%|███ | 30/100 [00:01<00:05, 13.17it/s, failures=0, objective=-1.07]
31%|███ | 31/100 [00:02<00:05, 13.17it/s, failures=0, objective=-1.07]
32%|███▏ | 32/100 [00:02<00:05, 11.77it/s, failures=0, objective=-1.07]
32%|███▏ | 32/100 [00:02<00:05, 11.77it/s, failures=0, objective=-1.07]
33%|███▎ | 33/100 [00:02<00:05, 11.77it/s, failures=0, objective=-1.07]
34%|███▍ | 34/100 [00:02<00:05, 11.38it/s, failures=0, objective=-1.07]
34%|███▍ | 34/100 [00:02<00:05, 11.38it/s, failures=0, objective=-1.07]
35%|███▌ | 35/100 [00:02<00:05, 11.38it/s, failures=0, objective=-1.07]
36%|███▌ | 36/100 [00:02<00:05, 11.00it/s, failures=0, objective=-1.07]
36%|███▌ | 36/100 [00:02<00:05, 11.00it/s, failures=0, objective=-1.07]
37%|███▋ | 37/100 [00:02<00:05, 11.00it/s, failures=0, objective=-1.07]
38%|███▊ | 38/100 [00:02<00:05, 10.71it/s, failures=0, objective=-1.07]
38%|███▊ | 38/100 [00:02<00:05, 10.71it/s, failures=0, objective=-1.07]
39%|███▉ | 39/100 [00:02<00:05, 10.71it/s, failures=0, objective=-1.07]
40%|████ | 40/100 [00:03<00:05, 10.42it/s, failures=0, objective=-1.07]
40%|████ | 40/100 [00:03<00:05, 10.42it/s, failures=0, objective=-1.07]
41%|████ | 41/100 [00:03<00:05, 10.42it/s, failures=0, objective=-1.07]
42%|████▏ | 42/100 [00:03<00:06, 9.44it/s, failures=0, objective=-1.07]
42%|████▏ | 42/100 [00:03<00:06, 9.44it/s, failures=0, objective=-1.07]
43%|████▎ | 43/100 [00:03<00:06, 9.44it/s, failures=0, objective=-1.07]
44%|████▍ | 44/100 [00:03<00:06, 9.31it/s, failures=0, objective=-1.07]
44%|████▍ | 44/100 [00:03<00:06, 9.31it/s, failures=0, objective=-1.07]
45%|████▌ | 45/100 [00:03<00:05, 9.28it/s, failures=0, objective=-1.07]
45%|████▌ | 45/100 [00:03<00:05, 9.28it/s, failures=0, objective=-0.174]
46%|████▌ | 46/100 [00:03<00:05, 9.26it/s, failures=0, objective=-0.174]
46%|████▌ | 46/100 [00:03<00:05, 9.26it/s, failures=0, objective=-0.174]
47%|████▋ | 47/100 [00:03<00:05, 9.25it/s, failures=0, objective=-0.174]
47%|████▋ | 47/100 [00:03<00:05, 9.25it/s, failures=0, objective=-0.174]
48%|████▊ | 48/100 [00:03<00:05, 9.23it/s, failures=0, objective=-0.174]
48%|████▊ | 48/100 [00:03<00:05, 9.23it/s, failures=0, objective=-0.174]
49%|████▉ | 49/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
49%|████▉ | 49/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
50%|█████ | 50/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
50%|█████ | 50/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
51%|█████ | 51/100 [00:04<00:05, 9.23it/s, failures=0, objective=-0.174]
51%|█████ | 51/100 [00:04<00:05, 9.23it/s, failures=0, objective=-0.174]
52%|█████▏ | 52/100 [00:04<00:05, 9.20it/s, failures=0, objective=-0.174]
52%|█████▏ | 52/100 [00:04<00:05, 9.20it/s, failures=0, objective=-0.174]
53%|█████▎ | 53/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
53%|█████▎ | 53/100 [00:04<00:05, 9.21it/s, failures=0, objective=-0.174]
54%|█████▍ | 54/100 [00:04<00:05, 9.20it/s, failures=0, objective=-0.174]
54%|█████▍ | 54/100 [00:04<00:05, 9.20it/s, failures=0, objective=-0.174]
55%|█████▌ | 55/100 [00:04<00:04, 9.15it/s, failures=0, objective=-0.174]
55%|█████▌ | 55/100 [00:04<00:04, 9.15it/s, failures=0, objective=-0.174]
56%|█████▌ | 56/100 [00:04<00:04, 9.13it/s, failures=0, objective=-0.174]
56%|█████▌ | 56/100 [00:04<00:04, 9.13it/s, failures=0, objective=-0.174]
57%|█████▋ | 57/100 [00:04<00:04, 9.17it/s, failures=0, objective=-0.174]
57%|█████▋ | 57/100 [00:04<00:04, 9.17it/s, failures=0, objective=-0.174]
58%|█████▊ | 58/100 [00:05<00:04, 9.14it/s, failures=0, objective=-0.174]
58%|█████▊ | 58/100 [00:05<00:04, 9.14it/s, failures=0, objective=-0.174]
59%|█████▉ | 59/100 [00:05<00:04, 9.11it/s, failures=0, objective=-0.174]
59%|█████▉ | 59/100 [00:05<00:04, 9.11it/s, failures=0, objective=-0.174]
60%|██████ | 60/100 [00:05<00:04, 9.12it/s, failures=0, objective=-0.174]
60%|██████ | 60/100 [00:05<00:04, 9.12it/s, failures=0, objective=-0.174]
61%|██████ | 61/100 [00:05<00:04, 9.12it/s, failures=0, objective=-0.174]
61%|██████ | 61/100 [00:05<00:04, 9.12it/s, failures=0, objective=-0.174]
62%|██████▏ | 62/100 [00:05<00:04, 9.04it/s, failures=0, objective=-0.174]
62%|██████▏ | 62/100 [00:05<00:04, 9.04it/s, failures=0, objective=-0.174]
63%|██████▎ | 63/100 [00:05<00:04, 9.02it/s, failures=0, objective=-0.174]
63%|██████▎ | 63/100 [00:05<00:04, 9.02it/s, failures=0, objective=-0.174]
64%|██████▍ | 64/100 [00:05<00:03, 9.05it/s, failures=0, objective=-0.174]
64%|██████▍ | 64/100 [00:05<00:03, 9.05it/s, failures=0, objective=-0.174]
65%|██████▌ | 65/100 [00:05<00:03, 9.04it/s, failures=0, objective=-0.174]
65%|██████▌ | 65/100 [00:05<00:03, 9.04it/s, failures=0, objective=-0.174]
66%|██████▌ | 66/100 [00:05<00:03, 9.04it/s, failures=0, objective=-0.174]
66%|██████▌ | 66/100 [00:05<00:03, 9.04it/s, failures=0, objective=-0.174]
67%|██████▋ | 67/100 [00:06<00:03, 9.06it/s, failures=0, objective=-0.174]
67%|██████▋ | 67/100 [00:06<00:03, 9.06it/s, failures=0, objective=-0.174]
68%|██████▊ | 68/100 [00:06<00:03, 9.05it/s, failures=0, objective=-0.174]
68%|██████▊ | 68/100 [00:06<00:03, 9.05it/s, failures=0, objective=-0.174]
69%|██████▉ | 69/100 [00:06<00:03, 9.03it/s, failures=0, objective=-0.174]
69%|██████▉ | 69/100 [00:06<00:03, 9.03it/s, failures=0, objective=-0.174]
70%|███████ | 70/100 [00:06<00:03, 9.04it/s, failures=0, objective=-0.174]
70%|███████ | 70/100 [00:06<00:03, 9.04it/s, failures=0, objective=-0.174]
71%|███████ | 71/100 [00:06<00:03, 9.05it/s, failures=0, objective=-0.174]
71%|███████ | 71/100 [00:06<00:03, 9.05it/s, failures=0, objective=-0.174]
72%|███████▏ | 72/100 [00:06<00:03, 9.03it/s, failures=0, objective=-0.174]
72%|███████▏ | 72/100 [00:06<00:03, 9.03it/s, failures=0, objective=-0.174]
73%|███████▎ | 73/100 [00:06<00:02, 9.00it/s, failures=0, objective=-0.174]
73%|███████▎ | 73/100 [00:06<00:02, 9.00it/s, failures=0, objective=-0.174]
74%|███████▍ | 74/100 [00:06<00:02, 9.00it/s, failures=0, objective=-0.174]
74%|███████▍ | 74/100 [00:06<00:02, 9.00it/s, failures=0, objective=-0.174]
75%|███████▌ | 75/100 [00:06<00:02, 8.99it/s, failures=0, objective=-0.174]
75%|███████▌ | 75/100 [00:06<00:02, 8.99it/s, failures=0, objective=-0.158]
76%|███████▌ | 76/100 [00:07<00:02, 8.92it/s, failures=0, objective=-0.158]
76%|███████▌ | 76/100 [00:07<00:02, 8.92it/s, failures=0, objective=-0.158]
77%|███████▋ | 77/100 [00:07<00:02, 8.90it/s, failures=0, objective=-0.158]
77%|███████▋ | 77/100 [00:07<00:02, 8.90it/s, failures=0, objective=-0.113]
78%|███████▊ | 78/100 [00:07<00:02, 8.87it/s, failures=0, objective=-0.113]
78%|███████▊ | 78/100 [00:07<00:02, 8.87it/s, failures=0, objective=-0.113]
79%|███████▉ | 79/100 [00:07<00:02, 8.89it/s, failures=0, objective=-0.113]
79%|███████▉ | 79/100 [00:07<00:02, 8.89it/s, failures=0, objective=-0.113]
80%|████████ | 80/100 [00:07<00:02, 8.87it/s, failures=0, objective=-0.113]
80%|████████ | 80/100 [00:07<00:02, 8.87it/s, failures=0, objective=-0.113]
81%|████████ | 81/100 [00:07<00:02, 8.89it/s, failures=0, objective=-0.113]
81%|████████ | 81/100 [00:07<00:02, 8.89it/s, failures=0, objective=-0.113]
82%|████████▏ | 82/100 [00:07<00:02, 8.91it/s, failures=0, objective=-0.113]
82%|████████▏ | 82/100 [00:07<00:02, 8.91it/s, failures=0, objective=-0.113]
83%|████████▎ | 83/100 [00:07<00:01, 8.89it/s, failures=0, objective=-0.113]
83%|████████▎ | 83/100 [00:07<00:01, 8.89it/s, failures=0, objective=-0.113]
84%|████████▍ | 84/100 [00:07<00:01, 8.86it/s, failures=0, objective=-0.113]
84%|████████▍ | 84/100 [00:07<00:01, 8.86it/s, failures=0, objective=-0.113]
85%|████████▌ | 85/100 [00:08<00:01, 8.83it/s, failures=0, objective=-0.113]
85%|████████▌ | 85/100 [00:08<00:01, 8.83it/s, failures=0, objective=-0.113]
86%|████████▌ | 86/100 [00:08<00:01, 7.73it/s, failures=0, objective=-0.113]
86%|████████▌ | 86/100 [00:08<00:01, 7.73it/s, failures=0, objective=-0.113]
87%|████████▋ | 87/100 [00:08<00:01, 7.97it/s, failures=0, objective=-0.113]
87%|████████▋ | 87/100 [00:08<00:01, 7.97it/s, failures=0, objective=-0.113]
88%|████████▊ | 88/100 [00:08<00:01, 8.21it/s, failures=0, objective=-0.113]
88%|████████▊ | 88/100 [00:08<00:01, 8.21it/s, failures=0, objective=-0.113]
89%|████████▉ | 89/100 [00:08<00:01, 8.41it/s, failures=0, objective=-0.113]
89%|████████▉ | 89/100 [00:08<00:01, 8.41it/s, failures=0, objective=-0.113]
90%|█████████ | 90/100 [00:08<00:01, 8.55it/s, failures=0, objective=-0.113]
90%|█████████ | 90/100 [00:08<00:01, 8.55it/s, failures=0, objective=-0.113]
91%|█████████ | 91/100 [00:08<00:01, 8.66it/s, failures=0, objective=-0.113]
91%|█████████ | 91/100 [00:08<00:01, 8.66it/s, failures=0, objective=-0.113]
92%|█████████▏| 92/100 [00:08<00:00, 8.72it/s, failures=0, objective=-0.113]
92%|█████████▏| 92/100 [00:08<00:00, 8.72it/s, failures=0, objective=-0.0033]
93%|█████████▎| 93/100 [00:09<00:00, 8.72it/s, failures=0, objective=-0.0033]
93%|█████████▎| 93/100 [00:09<00:00, 8.72it/s, failures=0, objective=-0.0033]
94%|█████████▍| 94/100 [00:09<00:00, 8.71it/s, failures=0, objective=-0.0033]
94%|█████████▍| 94/100 [00:09<00:00, 8.71it/s, failures=0, objective=-0.0033]
95%|█████████▌| 95/100 [00:09<00:00, 8.76it/s, failures=0, objective=-0.0033]
95%|█████████▌| 95/100 [00:09<00:00, 8.76it/s, failures=0, objective=-0.0033]
96%|█████████▌| 96/100 [00:09<00:00, 8.79it/s, failures=0, objective=-0.0033]
96%|█████████▌| 96/100 [00:09<00:00, 8.79it/s, failures=0, objective=-0.0033]
97%|█████████▋| 97/100 [00:09<00:00, 8.83it/s, failures=0, objective=-0.0033]
97%|█████████▋| 97/100 [00:09<00:00, 8.83it/s, failures=0, objective=-0.0033]
98%|█████████▊| 98/100 [00:09<00:00, 8.81it/s, failures=0, objective=-0.0033]
98%|█████████▊| 98/100 [00:09<00:00, 8.81it/s, failures=0, objective=-0.0033]
99%|█████████▉| 99/100 [00:09<00:00, 7.75it/s, failures=0, objective=-0.0033]
99%|█████████▉| 99/100 [00:09<00:00, 7.75it/s, failures=0, objective=-0.0033]
100%|██████████| 100/100 [00:09<00:00, 8.03it/s, failures=0, objective=-0.0033]
100%|██████████| 100/100 [00:09<00:00, 8.03it/s, failures=0, objective=-0.0033]
evaluator_large_tl = Evaluator.create(
run_large, method="thread", method_kwargs={"callbacks": [TqdmCallback()]}
)
search_large_tl = CBO(problem_large, evaluator_large_tl, random_state=42)
search_large_tl.fit_generative_model(results_small)
results["Large+TL"] = search_large_tl.search(max_evals)
/Users/romainegele/miniforge3/envs/dh-3.12-240724/lib/python3.12/site-packages/rdt/transformers/utils.py:12: DeprecationWarning: module 'sre_parse' is deprecated
import sre_parse # isort:skip
0%| | 0/100 [00:00<?, ?it/s]
1%| | 1/100 [00:00<00:00, 24385.49it/s, failures=0, objective=-58.7]
2%|▏ | 2/100 [00:00<00:07, 13.58it/s, failures=0, objective=-58.7]
2%|▏ | 2/100 [00:00<00:07, 13.58it/s, failures=0, objective=-58.7]
3%|▎ | 3/100 [00:00<00:07, 13.58it/s, failures=0, objective=-58.7]
4%|▍ | 4/100 [00:00<00:11, 8.43it/s, failures=0, objective=-58.7]
4%|▍ | 4/100 [00:00<00:11, 8.43it/s, failures=0, objective=-30.2]
5%|▌ | 5/100 [00:00<00:12, 7.70it/s, failures=0, objective=-30.2]
5%|▌ | 5/100 [00:00<00:12, 7.70it/s, failures=0, objective=-30.2]
6%|▌ | 6/100 [00:00<00:12, 7.36it/s, failures=0, objective=-30.2]
6%|▌ | 6/100 [00:00<00:12, 7.36it/s, failures=0, objective=-30.2]
7%|▋ | 7/100 [00:00<00:12, 7.17it/s, failures=0, objective=-30.2]
7%|▋ | 7/100 [00:00<00:12, 7.17it/s, failures=0, objective=-30.2]
8%|▊ | 8/100 [00:01<00:13, 7.01it/s, failures=0, objective=-30.2]
8%|▊ | 8/100 [00:01<00:13, 7.01it/s, failures=0, objective=-30.2]
9%|▉ | 9/100 [00:01<00:13, 6.91it/s, failures=0, objective=-30.2]
9%|▉ | 9/100 [00:01<00:13, 6.91it/s, failures=0, objective=-30.2]
10%|█ | 10/100 [00:01<00:13, 6.81it/s, failures=0, objective=-30.2]
10%|█ | 10/100 [00:01<00:13, 6.81it/s, failures=0, objective=-1.84]
11%|█ | 11/100 [00:01<00:15, 5.87it/s, failures=0, objective=-1.84]
11%|█ | 11/100 [00:01<00:15, 5.87it/s, failures=0, objective=-1.84]
12%|█▏ | 12/100 [00:01<00:16, 5.32it/s, failures=0, objective=-1.84]
12%|█▏ | 12/100 [00:01<00:16, 5.32it/s, failures=0, objective=-1.84]
13%|█▎ | 13/100 [00:02<00:17, 5.00it/s, failures=0, objective=-1.84]
13%|█▎ | 13/100 [00:02<00:17, 5.00it/s, failures=0, objective=-1.84]
14%|█▍ | 14/100 [00:02<00:17, 4.80it/s, failures=0, objective=-1.84]
14%|█▍ | 14/100 [00:02<00:17, 4.80it/s, failures=0, objective=-1.58]
15%|█▌ | 15/100 [00:02<00:18, 4.63it/s, failures=0, objective=-1.58]
15%|█▌ | 15/100 [00:02<00:18, 4.63it/s, failures=0, objective=-1.58]
16%|█▌ | 16/100 [00:02<00:18, 4.53it/s, failures=0, objective=-1.58]
16%|█▌ | 16/100 [00:02<00:18, 4.53it/s, failures=0, objective=-0.142]
17%|█▋ | 17/100 [00:03<00:22, 3.72it/s, failures=0, objective=-0.142]
17%|█▋ | 17/100 [00:03<00:22, 3.72it/s, failures=0, objective=-0.142]
18%|█▊ | 18/100 [00:03<00:21, 3.83it/s, failures=0, objective=-0.142]
18%|█▊ | 18/100 [00:03<00:21, 3.83it/s, failures=0, objective=-0.142]
19%|█▉ | 19/100 [00:03<00:20, 3.90it/s, failures=0, objective=-0.142]
19%|█▉ | 19/100 [00:03<00:20, 3.90it/s, failures=0, objective=-0.142]
20%|██ | 20/100 [00:03<00:20, 3.97it/s, failures=0, objective=-0.142]
20%|██ | 20/100 [00:03<00:20, 3.97it/s, failures=0, objective=-0.142]
21%|██ | 21/100 [00:04<00:19, 4.01it/s, failures=0, objective=-0.142]
21%|██ | 21/100 [00:04<00:19, 4.01it/s, failures=0, objective=-0.142]
22%|██▏ | 22/100 [00:04<00:19, 4.05it/s, failures=0, objective=-0.142]
22%|██▏ | 22/100 [00:04<00:19, 4.05it/s, failures=0, objective=-0.142]
100%|██████████| 100/100 [00:24<00:00, 11.59it/s, failures=0, objective=-6.91e-10]
23%|██▎ | 23/100 [00:04<00:18, 4.06it/s, failures=0, objective=-0.142]
23%|██▎ | 23/100 [00:04<00:18, 4.06it/s, failures=0, objective=-0.09]
24%|██▍ | 24/100 [00:04<00:19, 3.82it/s, failures=0, objective=-0.09]
24%|██▍ | 24/100 [00:04<00:19, 3.82it/s, failures=0, objective=-0.09]
25%|██▌ | 25/100 [00:05<00:19, 3.87it/s, failures=0, objective=-0.09]
25%|██▌ | 25/100 [00:05<00:19, 3.87it/s, failures=0, objective=-0.09]
26%|██▌ | 26/100 [00:05<00:18, 3.93it/s, failures=0, objective=-0.09]
26%|██▌ | 26/100 [00:05<00:18, 3.93it/s, failures=0, objective=-0.0869]
27%|██▋ | 27/100 [00:05<00:18, 3.97it/s, failures=0, objective=-0.0869]
27%|██▋ | 27/100 [00:05<00:18, 3.97it/s, failures=0, objective=-0.0869]
28%|██▊ | 28/100 [00:05<00:18, 4.00it/s, failures=0, objective=-0.0869]
28%|██▊ | 28/100 [00:05<00:18, 4.00it/s, failures=0, objective=-0.0869]
29%|██▉ | 29/100 [00:06<00:17, 4.04it/s, failures=0, objective=-0.0869]
29%|██▉ | 29/100 [00:06<00:17, 4.04it/s, failures=0, objective=-0.0302]
30%|███ | 30/100 [00:06<00:17, 4.07it/s, failures=0, objective=-0.0302]
30%|███ | 30/100 [00:06<00:17, 4.07it/s, failures=0, objective=-0.0302]
31%|███ | 31/100 [00:06<00:16, 4.07it/s, failures=0, objective=-0.0302]
31%|███ | 31/100 [00:06<00:16, 4.07it/s, failures=0, objective=-0.0302]
32%|███▏ | 32/100 [00:06<00:18, 3.63it/s, failures=0, objective=-0.0302]
32%|███▏ | 32/100 [00:06<00:18, 3.63it/s, failures=0, objective=-0.0302]
33%|███▎ | 33/100 [00:07<00:17, 3.75it/s, failures=0, objective=-0.0302]
33%|███▎ | 33/100 [00:07<00:17, 3.75it/s, failures=0, objective=-0.0302]
34%|███▍ | 34/100 [00:07<00:17, 3.84it/s, failures=0, objective=-0.0302]
34%|███▍ | 34/100 [00:07<00:17, 3.84it/s, failures=0, objective=-0.0302]
35%|███▌ | 35/100 [00:07<00:16, 3.90it/s, failures=0, objective=-0.0302]
35%|███▌ | 35/100 [00:07<00:16, 3.90it/s, failures=0, objective=-0.000943]
36%|███▌ | 36/100 [00:07<00:16, 3.95it/s, failures=0, objective=-0.000943]
36%|███▌ | 36/100 [00:07<00:16, 3.95it/s, failures=0, objective=-0.000943]
37%|███▋ | 37/100 [00:08<00:15, 3.96it/s, failures=0, objective=-0.000943]
37%|███▋ | 37/100 [00:08<00:15, 3.96it/s, failures=0, objective=-0.000943]
38%|███▊ | 38/100 [00:08<00:15, 3.99it/s, failures=0, objective=-0.000943]
38%|███▊ | 38/100 [00:08<00:15, 3.99it/s, failures=0, objective=-0.000943]
39%|███▉ | 39/100 [00:08<00:15, 4.01it/s, failures=0, objective=-0.000943]
39%|███▉ | 39/100 [00:08<00:15, 4.01it/s, failures=0, objective=-0.000943]
40%|████ | 40/100 [00:08<00:15, 3.87it/s, failures=0, objective=-0.000943]
40%|████ | 40/100 [00:08<00:15, 3.87it/s, failures=0, objective=-0.000943]
41%|████ | 41/100 [00:09<00:15, 3.92it/s, failures=0, objective=-0.000943]
41%|████ | 41/100 [00:09<00:15, 3.92it/s, failures=0, objective=-0.000943]
42%|████▏ | 42/100 [00:09<00:14, 3.95it/s, failures=0, objective=-0.000943]
42%|████▏ | 42/100 [00:09<00:14, 3.95it/s, failures=0, objective=-0.000943]
43%|████▎ | 43/100 [00:09<00:14, 3.98it/s, failures=0, objective=-0.000943]
43%|████▎ | 43/100 [00:09<00:14, 3.98it/s, failures=0, objective=-0.000943]
44%|████▍ | 44/100 [00:09<00:14, 4.00it/s, failures=0, objective=-0.000943]
44%|████▍ | 44/100 [00:09<00:14, 4.00it/s, failures=0, objective=-0.000943]
45%|████▌ | 45/100 [00:10<00:13, 4.01it/s, failures=0, objective=-0.000943]
45%|████▌ | 45/100 [00:10<00:13, 4.01it/s, failures=0, objective=-0.000943]
46%|████▌ | 46/100 [00:10<00:13, 3.97it/s, failures=0, objective=-0.000943]
46%|████▌ | 46/100 [00:10<00:13, 3.97it/s, failures=0, objective=-0.000943]
47%|████▋ | 47/100 [00:10<00:13, 3.84it/s, failures=0, objective=-0.000943]
47%|████▋ | 47/100 [00:10<00:13, 3.84it/s, failures=0, objective=-0.000943]
48%|████▊ | 48/100 [00:10<00:13, 3.88it/s, failures=0, objective=-0.000943]
48%|████▊ | 48/100 [00:10<00:13, 3.88it/s, failures=0, objective=-0.000943]
49%|████▉ | 49/100 [00:11<00:13, 3.91it/s, failures=0, objective=-0.000943]
49%|████▉ | 49/100 [00:11<00:13, 3.91it/s, failures=0, objective=-0.000943]
50%|█████ | 50/100 [00:11<00:12, 3.95it/s, failures=0, objective=-0.000943]
50%|█████ | 50/100 [00:11<00:12, 3.95it/s, failures=0, objective=-0.000943]
51%|█████ | 51/100 [00:11<00:12, 3.97it/s, failures=0, objective=-0.000943]
51%|█████ | 51/100 [00:11<00:12, 3.97it/s, failures=0, objective=-0.000943]
52%|█████▏ | 52/100 [00:11<00:12, 3.97it/s, failures=0, objective=-0.000943]
52%|█████▏ | 52/100 [00:11<00:12, 3.97it/s, failures=0, objective=-0.000943]
53%|█████▎ | 53/100 [00:12<00:11, 3.99it/s, failures=0, objective=-0.000943]
53%|█████▎ | 53/100 [00:12<00:11, 3.99it/s, failures=0, objective=-0.000943]
54%|█████▍ | 54/100 [00:12<00:11, 4.00it/s, failures=0, objective=-0.000943]
54%|█████▍ | 54/100 [00:12<00:11, 4.00it/s, failures=0, objective=-0.000943]
55%|█████▌ | 55/100 [00:12<00:11, 4.00it/s, failures=0, objective=-0.000943]
55%|█████▌ | 55/100 [00:12<00:11, 4.00it/s, failures=0, objective=-0.000943]
56%|█████▌ | 56/100 [00:12<00:10, 4.01it/s, failures=0, objective=-0.000943]
56%|█████▌ | 56/100 [00:12<00:10, 4.01it/s, failures=0, objective=-0.000943]
57%|█████▋ | 57/100 [00:13<00:10, 4.02it/s, failures=0, objective=-0.000943]
57%|█████▋ | 57/100 [00:13<00:10, 4.02it/s, failures=0, objective=-0.000943]
58%|█████▊ | 58/100 [00:13<00:12, 3.38it/s, failures=0, objective=-0.000943]
58%|█████▊ | 58/100 [00:13<00:12, 3.38it/s, failures=0, objective=-0.000943]
59%|█████▉ | 59/100 [00:13<00:11, 3.49it/s, failures=0, objective=-0.000943]
59%|█████▉ | 59/100 [00:13<00:11, 3.49it/s, failures=0, objective=-0.000943]
60%|██████ | 60/100 [00:14<00:11, 3.58it/s, failures=0, objective=-0.000943]
60%|██████ | 60/100 [00:14<00:11, 3.58it/s, failures=0, objective=-0.000943]
61%|██████ | 61/100 [00:14<00:10, 3.64it/s, failures=0, objective=-0.000943]
61%|██████ | 61/100 [00:14<00:10, 3.64it/s, failures=0, objective=-0.000943]
62%|██████▏ | 62/100 [00:14<00:10, 3.70it/s, failures=0, objective=-0.000943]
62%|██████▏ | 62/100 [00:14<00:10, 3.70it/s, failures=0, objective=-8.53e-5]
63%|██████▎ | 63/100 [00:14<00:09, 3.75it/s, failures=0, objective=-8.53e-5]
63%|██████▎ | 63/100 [00:14<00:09, 3.75it/s, failures=0, objective=-8.53e-5]
64%|██████▍ | 64/100 [00:15<00:09, 3.82it/s, failures=0, objective=-8.53e-5]
64%|██████▍ | 64/100 [00:15<00:09, 3.82it/s, failures=0, objective=-8.53e-5]
65%|██████▌ | 65/100 [00:15<00:09, 3.86it/s, failures=0, objective=-8.53e-5]
65%|██████▌ | 65/100 [00:15<00:09, 3.86it/s, failures=0, objective=-8.53e-5]
66%|██████▌ | 66/100 [00:15<00:08, 3.90it/s, failures=0, objective=-8.53e-5]
66%|██████▌ | 66/100 [00:15<00:08, 3.90it/s, failures=0, objective=-8.53e-5]
67%|██████▋ | 67/100 [00:15<00:08, 3.82it/s, failures=0, objective=-8.53e-5]
67%|██████▋ | 67/100 [00:15<00:08, 3.82it/s, failures=0, objective=-8.53e-5]
68%|██████▊ | 68/100 [00:16<00:08, 3.87it/s, failures=0, objective=-8.53e-5]
68%|██████▊ | 68/100 [00:16<00:08, 3.87it/s, failures=0, objective=-8.53e-5]
69%|██████▉ | 69/100 [00:16<00:07, 3.89it/s, failures=0, objective=-8.53e-5]
69%|██████▉ | 69/100 [00:16<00:07, 3.89it/s, failures=0, objective=-8.53e-5]
70%|███████ | 70/100 [00:16<00:07, 3.92it/s, failures=0, objective=-8.53e-5]
70%|███████ | 70/100 [00:16<00:07, 3.92it/s, failures=0, objective=-8.53e-5]
71%|███████ | 71/100 [00:16<00:07, 3.95it/s, failures=0, objective=-8.53e-5]
71%|███████ | 71/100 [00:16<00:07, 3.95it/s, failures=0, objective=-8.53e-5]
72%|███████▏ | 72/100 [00:17<00:07, 3.97it/s, failures=0, objective=-8.53e-5]
72%|███████▏ | 72/100 [00:17<00:07, 3.97it/s, failures=0, objective=-8.53e-5]
73%|███████▎ | 73/100 [00:17<00:06, 3.99it/s, failures=0, objective=-8.53e-5]
73%|███████▎ | 73/100 [00:17<00:06, 3.99it/s, failures=0, objective=-8.53e-5]
74%|███████▍ | 74/100 [00:17<00:06, 3.97it/s, failures=0, objective=-8.53e-5]
74%|███████▍ | 74/100 [00:17<00:06, 3.97it/s, failures=0, objective=-8.53e-5]
75%|███████▌ | 75/100 [00:17<00:06, 3.94it/s, failures=0, objective=-8.53e-5]
75%|███████▌ | 75/100 [00:17<00:06, 3.94it/s, failures=0, objective=-8.53e-5]
76%|███████▌ | 76/100 [00:18<00:06, 3.96it/s, failures=0, objective=-8.53e-5]
76%|███████▌ | 76/100 [00:18<00:06, 3.96it/s, failures=0, objective=-8.53e-5]
77%|███████▋ | 77/100 [00:18<00:05, 3.97it/s, failures=0, objective=-8.53e-5]
77%|███████▋ | 77/100 [00:18<00:05, 3.97it/s, failures=0, objective=-8.53e-5]
78%|███████▊ | 78/100 [00:18<00:05, 3.99it/s, failures=0, objective=-8.53e-5]
78%|███████▊ | 78/100 [00:18<00:05, 3.99it/s, failures=0, objective=-8.53e-5]
79%|███████▉ | 79/100 [00:18<00:05, 3.99it/s, failures=0, objective=-8.53e-5]
79%|███████▉ | 79/100 [00:18<00:05, 3.99it/s, failures=0, objective=-8.53e-5]
80%|████████ | 80/100 [00:19<00:05, 3.97it/s, failures=0, objective=-8.53e-5]
80%|████████ | 80/100 [00:19<00:05, 3.97it/s, failures=0, objective=-8.53e-5]
81%|████████ | 81/100 [00:19<00:05, 3.53it/s, failures=0, objective=-8.53e-5]
81%|████████ | 81/100 [00:19<00:05, 3.53it/s, failures=0, objective=-8.53e-5]
82%|████████▏ | 82/100 [00:19<00:04, 3.65it/s, failures=0, objective=-8.53e-5]
82%|████████▏ | 82/100 [00:19<00:04, 3.65it/s, failures=0, objective=-8.53e-5]
83%|████████▎ | 83/100 [00:20<00:04, 3.74it/s, failures=0, objective=-8.53e-5]
83%|████████▎ | 83/100 [00:20<00:04, 3.74it/s, failures=0, objective=-8.53e-5]
84%|████████▍ | 84/100 [00:20<00:04, 3.81it/s, failures=0, objective=-8.53e-5]
84%|████████▍ | 84/100 [00:20<00:04, 3.81it/s, failures=0, objective=-8.53e-5]
85%|████████▌ | 85/100 [00:20<00:03, 3.87it/s, failures=0, objective=-8.53e-5]
85%|████████▌ | 85/100 [00:20<00:03, 3.87it/s, failures=0, objective=-8.53e-5]
86%|████████▌ | 86/100 [00:20<00:03, 3.90it/s, failures=0, objective=-8.53e-5]
86%|████████▌ | 86/100 [00:20<00:03, 3.90it/s, failures=0, objective=-8.53e-5]
87%|████████▋ | 87/100 [00:21<00:03, 3.93it/s, failures=0, objective=-8.53e-5]
87%|████████▋ | 87/100 [00:21<00:03, 3.93it/s, failures=0, objective=-8.53e-5]
88%|████████▊ | 88/100 [00:21<00:03, 3.91it/s, failures=0, objective=-8.53e-5]
88%|████████▊ | 88/100 [00:21<00:03, 3.91it/s, failures=0, objective=-8.53e-5]
89%|████████▉ | 89/100 [00:21<00:02, 3.93it/s, failures=0, objective=-8.53e-5]
89%|████████▉ | 89/100 [00:21<00:02, 3.93it/s, failures=0, objective=-8.53e-5]
90%|█████████ | 90/100 [00:21<00:02, 3.95it/s, failures=0, objective=-8.53e-5]
90%|█████████ | 90/100 [00:21<00:02, 3.95it/s, failures=0, objective=-8.53e-5]
91%|█████████ | 91/100 [00:22<00:02, 3.96it/s, failures=0, objective=-8.53e-5]
91%|█████████ | 91/100 [00:22<00:02, 3.96it/s, failures=0, objective=-8.53e-5]
92%|█████████▏| 92/100 [00:22<00:02, 3.97it/s, failures=0, objective=-8.53e-5]
92%|█████████▏| 92/100 [00:22<00:02, 3.97it/s, failures=0, objective=-8.53e-5]
93%|█████████▎| 93/100 [00:22<00:01, 3.98it/s, failures=0, objective=-8.53e-5]
93%|█████████▎| 93/100 [00:22<00:01, 3.98it/s, failures=0, objective=-8.53e-5]
94%|█████████▍| 94/100 [00:22<00:01, 3.97it/s, failures=0, objective=-8.53e-5]
94%|█████████▍| 94/100 [00:22<00:01, 3.97it/s, failures=0, objective=-8.53e-5]
95%|█████████▌| 95/100 [00:23<00:01, 3.98it/s, failures=0, objective=-8.53e-5]
95%|█████████▌| 95/100 [00:23<00:01, 3.98it/s, failures=0, objective=-8.53e-5]
96%|█████████▌| 96/100 [00:23<00:01, 3.97it/s, failures=0, objective=-8.53e-5]
96%|█████████▌| 96/100 [00:23<00:01, 3.97it/s, failures=0, objective=-8.53e-5]
97%|█████████▋| 97/100 [00:23<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
97%|█████████▋| 97/100 [00:23<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
98%|█████████▊| 98/100 [00:23<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
98%|█████████▊| 98/100 [00:23<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
99%|█████████▉| 99/100 [00:24<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
99%|█████████▉| 99/100 [00:24<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
100%|██████████| 100/100 [00:24<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
100%|██████████| 100/100 [00:24<00:00, 3.97it/s, failures=0, objective=-8.53e-5]
Finally, we compare the results and quickly see that transfer-learning provided a consequant speed-up for the search:
fig, ax = plt.subplots(figsize=figure_size(width=600))
for strategy, df in results.items():
plot_search_trajectory_single_objective_hpo(
df,
show_failures=False,
mode="min",
ax=ax,
label=strategy,
)
plt.xlabel("Time (sec.)")
plt.ylabel("Objective")
plt.yscale("log")
plt.grid(visible=True, which="minor", linestyle=":")
plt.grid(visible=True, which="major", linestyle="-")
plt.legend()
plt.show()
Total running time of the script: (0 minutes 45.074 seconds)