deephyper.hpo.RandomSearch

deephyper.hpo.RandomSearch#

class deephyper.hpo.RandomSearch(problem, random_state=None, log_dir='.', verbose=0, stopper=None, checkpoint_history_to_csv: bool = True, solution_selection: Literal['argmax_obs', 'argmax_est'] | SolutionSelection | None = None)[source]#

Bases: Search

Random search algorithm used as an example for the API to implement new search algorithms.

Single-Objective

Multi-Objectives

Failures

✅

✅

✅
Parameters:

  • problem – object describing the search/optimization problem.

  • random_state (np.random.RandomState, optional) – Initial random state of the search. Defaults to None.

  • log_dir (str, optional) – Path to the directoy where results of the search are stored. Defaults to ".".

  • verbose (int, optional) – Use verbose mode. Defaults to 0.

  • stopper (Stopper, optional) – a stopper to leverage multi-fidelity when evaluating the function. Defaults to None which does not use any stopper.

  • checkpoint_history_to_csv (bool, optional) – wether the results from progressively collected evaluations should be checkpointed regularly to disc as a csv. Defaults to True.

  • solution_selection (Literal["argmax_obs", "argmax_est"] | SolutionSelection, optional) – the solution selection strategy. It can be a string where "argmax_obs" would select the argmax of observed objective values, and "argmax_est" would select the argmax of estimated objective values (through a predictive model).

Methods

ask

Ask the search for new configurations to evaluate.

check_evaluator

Check if the input is a callable, an evaluator or else.

dump_jobs_done_to_csv

Dump jobs completed to CSV in log_dir.

get_params

Get parameters used for the search object.

reload_checkpoint

save_params

Save the search parameters to a JSON file in the log folder.

search

Execute the search algorithm.

tell

Tell the search the results of the evaluations.

Attributes

search_id

The identifier of the search used by the evaluator.
ask(n: int = 1) → List[Dict]#

Ask the search for new configurations to evaluate.

Parameters:

n (int, optional) – The number of configurations to ask. Defaults to 1.

Returns:

a list of hyperparameter configurations to evaluate.

Return type:

List[Dict]

check_evaluator(evaluator)#

Check if the input is a callable, an evaluator or else.

dump_jobs_done_to_csv(flush: bool = False)#

Dump jobs completed to CSV in log_dir.

Parameters:

flush (bool, optional) – Force the dumping if set to True. Defaults to False.

get_params() → dict[str, Any]#

Get parameters used for the search object.

Returns:

A dictionary of the search parameters.

save_params(filename: str = 'params.json')#

Save the search parameters to a JSON file in the log folder.

Parameters:

filename – Name of JSON file where search parameters are saved. Default is params.json.

search(evaluator, max_evals: int = -1, timeout: int | float | None = None, max_evals_strict: bool = False) → DataFrame#

Execute the search algorithm.

Parameters:

  • evaluator – object describing the evaluation process.

  • max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.

  • timeout (int, optional) – The time budget (in seconds) of the search before stopping. Defaults to None, will not impose a time budget.

  • max_evals_strict (bool, optional) – If True the search will not spawn more than max_evals jobs. Defaults to False.

Returns:

A pandas DataFrame containing the evaluations performed or None if the

search could not evaluate any configuration.

This DataFrame contains the following columns: - p:HYPERPARAMETER_NAME: for each hyperparameter of the problem. - objective: for single objective optimization. - objective_0, objective_1, …: for multi-objective optimization. - job_id: the identifier of the job. - job_status: the status of the job at the end of the search. - m:METADATA_NAME: for each metadata of the problem. Some metadata are always

present like m:timestamp_submit and m:timestamp_gather which are the timestamps of the submission and gathering of the job.

Return type:

pd.DataFrame

property search_id#

The identifier of the search used by the evaluator.

tell(results: list[tuple[dict[str, str | int | float | None], str | int | float | tuple[str | int | float]]])#

Tell the search the results of the evaluations.

Parameters:

results (list[tuple[dict[str, Optional[str | int | float]], str | int | float]]) – a dictionary containing the results of the evaluations.

Contents