deephyper.search.hps.DBO#

class deephyper.search.hps.DBO(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, surrogate_model='RF', acq_func: str = 'UCB', acq_optimizer: str = 'auto', kappa: float = 1.96, xi: float = 0.001, n_points: int = 10000, filter_duplicated: bool = True, update_prior: bool = False, multi_point_strategy: str = 'cl_max', n_jobs: int = 1, n_initial_points: int = 10, initial_point_generator: str = 'random', initial_points=None, sync_communication: bool = False, filter_failures: str = 'mean', max_failures: int = 100, moo_scalarization_strategy: str = 'Chebyshev', moo_scalarization_weight=None, scheduler=None, **kwargs)[source]#

Bases: deephyper.search.hps._cbo.CBO

Distributed Bayesian Optimization Search.

Parameters

problem (HpProblem) – Hyperparameter problem describing the search space to explore.
run_function (callable) – A callable instance which represents the black-box function we want to evaluate.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to ".".
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
comm (optional) – The MPI communicator to use. Defaults to None.
run_function_kwargs (dict) – Keyword arguments to pass to the run-function. Defaults to None.
n_jobs (int, optional) – Parallel processes per rank to use for optimization updates (e.g., model re-fitting). Not used in surrogate_model if passed as own sklearn regressor. Defaults to 1.
surrogate_model (Union[str,sklearn.base.RegressorMixin], optional) – Type of the surrogate model to use. Can be a value in ["RF", "GP", "ET", "GBRT", "DUMMY"] or a sklearn regressor. "DUMMY" can be used of random-search, "GP" for Gaussian-Process (efficient with few iterations such as a hundred sequentially but bottleneck when scaling because of its cubic complexity w.r.t. the number of evaluations), “``”RF”`` for the Random-Forest regressor (log-linear complexity with respect to the number of evaluations). Defaults to "RF".
n_initial_points (int, optional) – Number of collected objectives required before fitting the surrogate-model. Defaults to 10.
initial_point_generator (str, optional) – Sets an initial points generator. Can be either ["random", "sobol", "halton", "hammersly", "lhs", "grid"]. Defaults to "random".
lazy_socket_allocation (bool, optional) – If True then MPI communication socket are initialized only when used for the first time, otherwise the initialization is forced when creating the instance. Defaults to False.
sync_communication (bool, optional) – If True workers communicate synchronously, otherwise workers communicate asynchronously. Defaults to False.
sync_communication_freq (int, optional) – Manage the frequency at which workers should communicate their results in the case of synchronous communication. Defaults to 10.
checkpoint_file (str) – Name of the file in log_dir where results are checkpointed. Defaults to "results.csv".
checkpoint_freq (int) – Frequency at which results are checkpointed. Defaults to 1.
acq_func (str) – Acquisition function to use. If "UCB" then the upper confidence bound is used, if "EI" then the expected-improvement is used, if "PI" then the probability of improvement is used, if "gp_hedge" then probabilistically choose one of the above.
acq_optimizer (str) – Method use to optimise the acquisition function. If "sampling" then random-samples are drawn and infered for optimization, if "lbfgs" gradient-descent is used. Defaults to "auto".
kappa (float) – Exploration/exploitation value for UCB-acquisition function, the higher the more exploration, the smaller the more exploitation. Defaults to 1.96 which corresponds to a 95% confidence interval.
xi (float) – Exploration/exploitation value for EI and PI-acquisition functions, the higher the more exploration, the smaller the more exploitation. Defaults to 0.001.
sample_max_size (int) – Maximum size of the number of samples used to re-fit the surrogate model. Defaults to -1 for infinite sample size.
sample_strategy (str) – Sub-sampling strategy to re-fit the surrogate model. If "quantile" then sub-sampling is performed based on the quantile of the collected objective values. Defaults to "quantile".

Methods

`check_evaluator`
`dump_context`	Dumps the context in the log folder.
`fit_generative_model`	Learn the distribution of hyperparameters for the top-`(1-q)x100%` configurations and sample from this distribution.
`fit_search_space`	Apply prior-guided transfer learning based on a DataFrame of results.
`fit_surrogate`	Fit the surrogate model of the search from a checkpointed Dataframe.
`search`	Execute the search algorithm.
`to_json`	Returns a json version of the search object.

dump_context()#: Dumps the context in the log folder.

fit_generative_model(df, q=0.9, n_iter_optimize=0, n_samples=100)#

Learn the distribution of hyperparameters for the top-(1-q)x100% configurations and sample from this distribution. It can be used for transfer learning. For multiobjective problems, this function computes the top-(1-q)x100% configurations in terms of their ranking with respect to pareto efficiency: all points on the first non-dominated pareto front have rank 1 and in general, points on the k’th non-dominated front have rank k.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

Parameters

df (str|DataFrame) – a dataframe or path to CSV from a previous search.
q (float, optional) – the quantile defined the set of top configurations used to bias the search. Defaults to 0.90 which select the top-10% configurations from df.
n_iter_optimize (int, optional) – the number of iterations used to optimize the generative model which samples the data for the search. Defaults to 0 with no optimization for the generative model.
n_samples (int, optional) – the number of samples used to score the generative model.

Returns

score, model which are a metric which measures the quality of the learned generated-model and the generative model respectively.

Return type

tuple

fit_search_space(df, fac_numerical=0.125, fac_categorical=10)#

Apply prior-guided transfer learning based on a DataFrame of results.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

Parameters

df (str|DataFrame) – a checkpoint from a previous search.
fac_numerical (float) – the factor used to compute the sigma of a truncated normal distribution based on sigma = max(1.0, (upper - lower) * fac_numerical). A small large factor increase exploration while a small factor increase exploitation around the best-configuration from the df parameter.
fac_categorical (float) – the weight given to a categorical feature part of the best configuration. A large weight > 1 increase exploitation while a small factor close to 1 increase exploration.

fit_surrogate(df)#

Fit the surrogate model of the search from a checkpointed Dataframe.

Parameters: df (str|DataFrame) – a checkpoint from a previous search.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

search(max_evals: int = - 1, timeout: Optional[int] = None)#

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget (in seconds) of the search before stopping. Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed or None if the search could not evaluate any configuration.

Return type

DataFrame

to_json()#: Returns a json version of the search object.