.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "examples/examples_bbo/plot_experimental_design.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_examples_examples_bbo_plot_experimental_design.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_examples_examples_bbo_plot_experimental_design.py:


Generating Combinations of Parameters for Standard Experimental Designs
=======================================================================

**Author(s)**: Romain Egele.

This example demonstrates how to evaluate parameters following standard experimental
designs such as random design, factorial design (a.k.a., grid search) and quasi-monte-carlo
designs (e.g., lhs, sobol).

More specifically in this example we will show factorial design.

See `Design of experiments (Wikipedia) <https://en.wikipedia.org/wiki/Design_of_experiments>`_ to learn more about this topic.

.. GENERATED FROM PYTHON SOURCE LINES 15-28

.. dropdown:: Code (Import statements)

    .. code-block:: Python

        import os
        import shutil

        import matplotlib.pyplot as plt

        from deephyper.analysis._matplotlib import update_matplotlib_rc
        from deephyper.hpo import HpProblem
        from deephyper.hpo import ExperimentalDesignSearch


        update_matplotlib_rc()








.. GENERATED FROM PYTHON SOURCE LINES 29-35

We start by defining the search space of parameters. 
For the purpose of demonstration, we will define three variables of different "types":

- `x`: is a real parameter drawn from a Log Uniform distribution in order to uniformly draw small and large values from the defined range of values. Otherwise we would have low probability of testing values near the lower-bound.
- `y`: is a discrete parameter drawn from a Uniform distribution. The discrete type is infered from the Python type of the bounds `int`.
- `z`: is a categorical ordinal parameter.

.. GENERATED FROM PYTHON SOURCE LINES 35-42

.. code-block:: Python


    problem = HpProblem()
    problem.add_hyperparameter((0.0001, 100.0, "log-uniform"), "x")
    problem.add_hyperparameter((0, 100), "y")
    problem.add_hyperparameter([1, 2, 3], "z")
    problem





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    Configuration space object:
      Hyperparameters:
        x, Type: UniformFloat, Range: [0.0001, 100.0], Default: 0.1, on log-scale
        y, Type: UniformInteger, Range: [0, 100], Default: 50
        z, Type: Ordinal, Sequence: {1, 2, 3}, Default: 1




.. GENERATED FROM PYTHON SOURCE LINES 43-44

We define the black-box function that we want to evaluate with these parameters.

.. GENERATED FROM PYTHON SOURCE LINES 44-49

.. code-block:: Python


    def run(job):
        objective = job.parameters["x"] + job.parameters["y"] + job.parameters["z"]
        return objective








.. GENERATED FROM PYTHON SOURCE LINES 50-56

.. dropdown:: Code (Clean up legacy results)

    .. code-block:: Python


        log_dir = "eds_logs"
        if os.path.exists(log_dir):
            shutil.rmtree(log_dir)








.. GENERATED FROM PYTHON SOURCE LINES 57-61

Then, we define the search that will generate parameters. For standard experimental designs we use
the :class:`deephyper.hpo.ExperimentalDesignSearch` class. For a grid search, we set ``design="grid"``. 
It is good to note that the :class:`deephyper.evaluator.Evaluator` can also be used with this class to parallelize evaluations.
Also, it is important to set `n_points` and `max_evals` to the same value.

.. GENERATED FROM PYTHON SOURCE LINES 61-71

.. code-block:: Python


    max_evals = 200
    search = ExperimentalDesignSearch(
        problem, 
        n_points=max_evals, 
        design="grid", 
        log_dir=log_dir,
    )
    results = search.search(run, max_evals)








.. GENERATED FROM PYTHON SOURCE LINES 72-73

Finally, we plot the results from the collected DataFrame.

.. GENERATED FROM PYTHON SOURCE LINES 73-80

.. dropdown:: Code (Make plot)

    .. code-block:: Python


        fig, ax = plt.subplots()
        ax.scatter(results["p:x"], results["p:y"], c=results["p:z"], alpha=0.3)
        ax.set_xscale("log")
        _ = plt.xlabel("x")
        _ = plt.ylabel("y")



.. image-sg:: /examples/examples_bbo/images/sphx_glr_plot_experimental_design_001.png
   :alt: plot experimental design
   :srcset: /examples/examples_bbo/images/sphx_glr_plot_experimental_design_001.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 3.086 seconds)


.. _sphx_glr_download_examples_examples_bbo_plot_experimental_design.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_experimental_design.ipynb <plot_experimental_design.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_experimental_design.py <plot_experimental_design.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_experimental_design.zip <plot_experimental_design.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_