deephyper.search.hps.MPIDistributedBO#

class deephyper.search.hps.MPIDistributedBO(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, surrogate_model='RF', acq_func: str = 'UCB', acq_optimizer: str = 'auto', kappa: float = 1.96, xi: float = 0.001, n_points: int = 10000, filter_duplicated: bool = True, update_prior: bool = False, multi_point_strategy: str = 'cl_max', n_jobs: int = 1, n_initial_points: int = 10, initial_point_generator: str = 'random', initial_points=None, sync_communication: bool = False, filter_failures: str = 'mean', max_failures: int = 100, moo_scalarization_strategy: str = 'Chebyshev', moo_scalarization_weight=None, scheduler=None, objective_scaler='auto', stopper: Optional[deephyper.stopper._stopper.Stopper] = None, comm: Optional[mpi4py.MPI.Comm] = None, **kwargs)[source]#

Bases: deephyper.search.hps._cbo.CBO

Distributed Bayesian Optimization Search using MPI to launch parallel search instances.

Parameters

problem (HpProblem) – Hyperparameter problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to ".".
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
surrogate_model (Union[str,sklearn.base.RegressorMixin], optional) – Surrogate model used by the Bayesian optimization. Can be a value in ["RF", "GP", "ET", "GBRT", "DUMMY"] or a sklearn regressor. "RF" is for Random-Forest which is the best compromise between speed and quality when performing a lot of parallel evaluations, i.e., reaching more than hundreds of evaluations. "GP" is for Gaussian-Process which is the best choice when maximizing the quality of iteration but quickly slow down when reaching hundreds of evaluations, also it does not support conditional search space. "ET" is for Extra-Tree, faster than random forest but with worse mean estimate and poor uncertainty quantification capabilities. "GBRT" is for Gradient-Boosting Regression Tree, it has better mean estimate than other tree-based method worse uncertainty quantification capabilities and slower than "RF". Defaults to "RF".
acq_func (str, optional) – Acquisition function used by the Bayesian optimization. Can be a value in ["UCB", "EI", "PI", "gp_hedge"]. Defaults to "UCB".
acq_optimizer (str, optional) – Method used to minimze the acquisition function. Can be a value in ["sampling", "lbfgs"]. Defaults to "auto".
kappa (float, optional) – Manage the exploration/exploitation tradeoff for the “UCB” acquisition function. Defaults to 1.96 which corresponds to 95% of the confidence interval.
xi (float, optional) – Manage the exploration/exploitation tradeoff of "EI" and "PI" acquisition function. Defaults to 0.001.
n_points (int, optional) – The number of configurations sampled from the search space to infer each batch of new evaluated configurations.
filter_duplicated (bool, optional) – Force the optimizer to sample unique points until the search space is “exhausted” in the sens that no new unique points can be found given the sampling size n_points. Defaults to True.
multi_point_strategy (str, optional) – Definition of the constant value use for the Liar strategy. Can be a value in ["cl_min", "cl_mean", "cl_max", "qUCB"]. All "cl_..." strategies follow the constant-liar scheme, where if $N$ new points are requested, the surrogate model is re-fitted $N-1$ times with lies (respectively, the minimum, mean and maximum objective found so far; for multiple objectives, these are the minimum, mean and maximum of the individual objectives) to infer the acquisition function. Constant-Liar strategy have poor scalability because of this repeated re-fitting. The "qUCB" strategy is much more efficient by sampling a new $kappa$ value for each new requested point without re-fitting the model, but it is only compatible with acq_func == "UCB". Defaults to "cl_max".
n_jobs (int, optional) – Number of parallel processes used to fit the surrogate model of the Bayesian optimization. A value of -1 will use all available cores. Not used in surrogate_model if passed as own sklearn regressor. Defaults to 1.
n_initial_points (int, optional) – Number of collected objectives required before fitting the surrogate-model. Defaults to 10.
initial_point_generator (str, optional) – Sets an initial points generator. Can be either ["random", "sobol", "halton", "hammersly", "lhs", "grid"]. Defaults to "random".
initial_points (List[Dict], optional) – A list of initial points to evaluate where each point is a dictionnary where keys are names of hyperparameters and values their corresponding choice. Defaults to None for them to be generated randomly from the search space.
sync_communcation (bool, optional) – Performs the search in a batch-synchronous manner. Defaults to False for asynchronous updates.
filter_failures (str, optional) – Replace objective of failed configurations by "min" or "mean". If "ignore" is passed then failed configurations will be filtered-out and not passed to the surrogate model. For multiple objectives, failure of any single objective will lead to treating that configuration as failed and each of these multiple objective will be replaced by their individual "min" or "mean" of past configurations. Defaults to "mean" to replace by failed configurations by the running mean of objectives.
max_failures (int, optional) – Maximum number of failed configurations allowed before observing a valid objective value when filter_failures is not equal to "ignore". Defaults to 100.
moo_scalarization_strategy (str, optional) – Scalarization strategy used in multiobjective optimization. Can be a value in ["Linear", "Chebyshev", "AugChebyshev", "PBI", "Quadratic", "rLinear", "rChebyshev", "rAugChebyshev", "rPBI", "rQuadratic"]. Defaults to "Chebyshev".
moo_scalarization_weight (list, optional) – Scalarization weights to be used in multiobjective optimization with length equal to the number of objective functions. Defaults to None.
scheduler (dict, callable, optional) – a method to manage the the value of kappa, xi with iterations. Defaults to None which does not use any scheduler.
objective_scaler (str, optional) – a way to map the objective space to some other support for example to normalize it. Defaults to "auto" which automatically set it to “identity” for any surrogate model except “RF” which will use “minmaxlog”.
stopper (Stopper, optional) – a stopper to leverage multi-fidelity when evaluating the function. Defaults to None which does not use any stopper.
comm (Comm, optional) – communicator used with MPI. Defaults to None for COMM_WORLD.

Methods

`bootstrap_evaluator`
`check_evaluator`
`dump_context`	Dumps the context in the log folder.
`fit_generative_model`	Learn the distribution of hyperparameters for the top-`(1-q)x100%` configurations and sample from this distribution.
`fit_search_space`	Apply prior-guided transfer learning based on a DataFrame of results.
`fit_surrogate`	Fit the surrogate model of the search from a checkpointed Dataframe.
`search`	Execute the search algorithm.
`to_json`	Returns a json version of the search object.

Attributes

search_id

The identifier of the search used by the evaluator.

dump_context()#: Dumps the context in the log folder.

fit_generative_model(df, q=0.9, n_iter_optimize=0, n_samples=100)#

Learn the distribution of hyperparameters for the top-(1-q)x100% configurations and sample from this distribution. It can be used for transfer learning. For multiobjective problems, this function computes the top-(1-q)x100% configurations in terms of their ranking with respect to pareto efficiency: all points on the first non-dominated pareto front have rank 1 and in general, points on the k’th non-dominated front have rank k.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

Parameters

df (str|DataFrame) – a dataframe or path to CSV from a previous search.
q (float, optional) – the quantile defined the set of top configurations used to bias the search. Defaults to 0.90 which select the top-10% configurations from df.
n_iter_optimize (int, optional) – the number of iterations used to optimize the generative model which samples the data for the search. Defaults to 0 with no optimization for the generative model.
n_samples (int, optional) – the number of samples used to score the generative model.

Returns

score, model which are a metric which measures the quality of the learned generated-model and the generative model respectively.

Return type

tuple

fit_search_space(df, fac_numerical=0.125, fac_categorical=10)#

Apply prior-guided transfer learning based on a DataFrame of results.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

Parameters

df (str|DataFrame) – a checkpoint from a previous search.
fac_numerical (float) – the factor used to compute the sigma of a truncated normal distribution based on sigma = max(1.0, (upper - lower) * fac_numerical). A small large factor increase exploration while a small factor increase exploitation around the best-configuration from the df parameter.
fac_categorical (float) – the weight given to a categorical feature part of the best configuration. A large weight > 1 increase exploitation while a small factor close to 1 increase exploration.

fit_surrogate(df)#

Fit the surrogate model of the search from a checkpointed Dataframe.

Parameters: df (str|DataFrame) – a checkpoint from a previous search.

Example Usage:

>>> search = CBO(problem, evaluator)
>>> search.fit_surrogate("results.csv")

search(max_evals: int = - 1, timeout: Optional[int] = None)#

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget (in seconds) of the search before stopping. Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed or None if the search could not evaluate any configuration.

Return type

DataFrame

property search_id#: The identifier of the search used by the evaluator.

to_json()#: Returns a json version of the search object.