.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "examples/plot_transfer_learning_for_hps.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:here  to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_examples_plot_transfer_learning_for_hps.py: Transfer Learning for Hyperparameter Search =========================================== **Author(s)**: Romain Egele. In this example we present how to apply transfer-learning for hyperparameter search. Let's assume you have a bunch of similar tasks for example the search of neural networks hyperparameters for different datasets. You can easily imagine that close choices of hyperparameters can perform well these different datasets even if some light additional tuning can help improve the performance. Therefore, you can perform an expensive search once to then reuse the explored set of hyperparameters of thid search and bias the following search with it. Here, we will use a cheap to compute and easy to understand example where we maximise the :math:f(x) = -\sum_{i=0}^{n-1} function. In this case the size of the problem can be defined by the variable :math:n. We will start by optimizing the small-size problem where :math:n=1, then apply transfer-learning from to optimize the larger-size problem where :math:n=2 and visualize the difference if were not to apply transfer-learning on this larger problem instance. Let us start by defining the run-functions of the small and large scale problems: .. GENERATED FROM PYTHON SOURCE LINES 14-25 .. code-block:: default import functools def run(config: dict, N: int) -> float: y = -sum([config[f"x{i}"] ** 2 for i in range(N)]) return y run_small = functools.partial(run, N=1) run_large = functools.partial(run, N=2) .. GENERATED FROM PYTHON SOURCE LINES 26-27 Then, we can define the hyperparameter problem space based on :math:n .. GENERATED FROM PYTHON SOURCE LINES 27-36 .. code-block:: default from deephyper.problem import HpProblem N = 1 problem_small = HpProblem() for i in range(N): problem_small.add_hyperparameter((-10.0, 10.0), f"x{i}") problem_small .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Configuration space object: Hyperparameters: x0, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0 .. GENERATED FROM PYTHON SOURCE LINES 37-43 .. code-block:: default N = 2 problem_large = HpProblem() for i in range(N): problem_large.add_hyperparameter((-10.0, 10.0), f"x{i}") problem_large .. rst-class:: sphx-glr-script-out Out: .. code-block:: none Configuration space object: Hyperparameters: x0, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0 x1, Type: UniformFloat, Range: [-10.0, 10.0], Default: 0.0 .. GENERATED FROM PYTHON SOURCE LINES 44-45 Then, we define setup the search and execute it: .. GENERATED FROM PYTHON SOURCE LINES 45-57 .. code-block:: default from deephyper.evaluator import Evaluator from deephyper.evaluator.callback import TqdmCallback from deephyper.search.hps import CBO results = {} max_evals = 20 evaluator_small = Evaluator.create( run_small, method="serial", method_kwargs={"callbacks": [TqdmCallback(max_evals)]} ) search_small = CBO(problem_small, evaluator_small, random_state=42) results["Small"] = search_small.search(max_evals) .. rst-class:: sphx-glr-script-out Out: .. code-block:: none 0%| | 0/20 [00:00 .. container:: sphx-glr-download sphx-glr-download-jupyter :download:Download Jupyter notebook: plot_transfer_learning_for_hps.ipynb  .. only:: html .. rst-class:: sphx-glr-signature Gallery generated by Sphinx-Gallery `_