# 1. DeepHyper 101# In this tutorial, we present the basics of DeepHyper.

:

try:
import deephyper
print(deephyper.__version__)
except (ImportError, ModuleNotFoundError):
!pip install deephyper

0.6.0


## 1.1. Optimization Problem#

In the definition of our optimization problem we have two components:

1. black-box function that we want to optimize

2. the search space of input variables

### 1.1.1. Black-Box Function#

DeepHyper is developed to optimize black-box functions. Here, we define the function $$f(x) = - x ^ 2$$ that we want to maximise (the maximum being $$f(x=0) = 0$$ on $$I_x = [-10;10]$$). The black-box function f takes as input a config dictionary from which we retrieve the variables of interest.

:

def f(job):
return -job.parameters["x"] ** 2


### 1.1.2. Search Space of Input Variables#

In this example, we have only one variable $$x$$ for the black-box functin $$f$$. We empirically decide to optimize this variable $$x$$ on the interval $$I_x = [-10;10]$$. To do so we use the HpProblem from DeepHyper and add a real hyperparameter by using a tuple of two floats.

:

from deephyper.problem import HpProblem

problem = HpProblem()

# Define the variable you want to optimize

problem

:

Configuration space object:
Hyperparameters:
x, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0


## 1.2. Evaluator Interface#

DeepHyper uses an API called Evaluator to distribute the computation of black-box functions and adapt to different backends (e.g., threads, processes, MPI, Ray). An Evaluator object wraps the black-box function f that we want to optimize. Then a method parameter is used to select the backend and method_kwargs defines some available options of this backend.

Tip

The method="thread" provides parallel computation only if the black-box is releasing the global interpretor lock (GIL). Therefore, if you want parallelism in Jupyter notebooks you should use the Ray evaluator (method="ray") after installing Ray with pip install ray.

It is possible to define callbacks to extend the behaviour of Evaluator each time a function-evaluation is launched or completed. In this example we use the TqdmCallback to follow the completed evaluations and the evolution of the objective with a progress-bar.

:

from deephyper.evaluator import Evaluator
from deephyper.evaluator.callback import TqdmCallback

# define the evaluator to distribute the computation
evaluator = Evaluator.create(
f,
method_kwargs={
"num_workers": 4,
"callbacks": [TqdmCallback()]
},
)

print(f"Evaluator has {evaluator.num_workers} available worker{'' if evaluator.num_workers == 1 else 's'}")

Evaluator has 4 available workers

/Users/romainegele/Documents/Argonne/deephyper/deephyper/evaluator/_evaluator.py:127: UserWarning: Applying nest-asyncio patch for IPython Shell!
warnings.warn(


## 1.3. Search Algorithm#

The next step is to define the search algorithm that we want to use. Here, we choose CBO (Centralized Bayesian Optimization) which is a sampling based Bayesian optimization strategy. This algorithm has the advantage of being asynchronous thanks to a constant liar strategy which is crutial to keep a good utilization of the resources when the number of available workers increases.

:

from deephyper.search.hps import CBO

search = CBO(
problem,
evaluator,
acq_func="UCB",  # Acquisition function to Upper Confidence Bound
multi_point_strategy="qUCB",  # Fast Multi-point strategy with q-Upper Confidence Bound
n_jobs=2,  # Number of threads to fit surrogate models in parallel
)


Then, we can execute the search for a given number of iterations by using the search.search(max_evals=...). It is also possible to use the timeout parameter if one needs a specific time budget (e.g., restricted computational time in machine learning competitions, allocation time in HPC).

:

results = search.search(max_evals=100)


Finally, let us visualize the results. The search(...) returns a DataFrame also saved locally under results.csv (in case of crash we don’t want to lose the possibly expensive evaluations already performed).

The DataFrame contains as columns: 1. the optimized hyperparameters: such as x in our case. 2. the id of each evaluated function (increased incrementally following the order of created evaluations). 3. the objective maximised which directly match the results of the $$f$$-function in our example. 4. the objective maximised which directly match the results of the $$f$$-function in our example. 5. the time of creation/collection of each task timestamp_submit and timestamp_gather respectively (in secondes, since the creation of the Evaluator).

:

results

:

p:x objective job_id m:timestamp_submit m:timestamp_gather
0 3.241574 -1.050780e+01 3 0.088059 0.088764
1 3.401464 -1.156996e+01 2 0.088030 0.094780
2 9.761872 -9.529415e+01 0 0.087993 0.094980
3 -2.094117 -4.385325e+00 1 0.088019 0.095260
4 4.381632 -1.919870e+01 7 0.114489 0.115110
... ... ... ... ... ...
95 -0.004540 -2.060865e-05 95 3.757070 3.758207
96 -0.000577 -3.328177e-07 97 3.979956 3.980287
97 -0.003078 -9.475395e-06 96 3.979945 3.980701
98 -0.000577 -3.328177e-07 99 3.979969 3.980882
99 -0.000577 -3.328177e-07 98 3.979963 3.981050

100 rows × 5 columns

We can also plot the evolution of the objective to verify that we converge correctly toward $$0$$.

:

from deephyper.analysis.hpo import plot_search_trajectory_single_objective_hpo

fig, ax = plot_search_trajectory_single_objective_hpo(results) [ ]: