deephyper.search.nas¶

Neural architecture search algorithms.

class deephyper.search.nas.AMBSMixed(problem, evaluator, random_state=None, log_dir='.', verbose=0, surrogate_model: str = 'RF', acq_func: str = 'UCB', kappa: float = 1.96, xi: float = 0.001, n_points: int = 10000, liar_strategy: str = 'cl_max', n_jobs: int = 1, **kwargs)[source]¶

Bases: deephyper.search.nas._base.NeuralArchitectureSearch

Asynchronous Model-Based Search baised on the Scikit-Optimized Optimizer. It is extended to the case of joint hyperparameter and neural architecture search.

Parameters

problem (NaProblem) – Neural architecture search problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to “.”.
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
surrogate_model (str, optional) – Surrogate model used by the Bayesian optimization. Can be a value in ["RF", "ET", "GBRT", "DUMMY"]. Defaults to "RF".
acq_func (str, optional) – Acquisition function used by the Bayesian optimization. Can be a value in ["UCB", "EI", "PI", "gp_hedge"]. Defaults to "UCB".
kappa (float, optional) – Manage the exploration/exploitation tradeoff for the “UCB” acquisition function. Defaults to 1.96 for a balance between exploitation and exploration.
xi (float, optional) – Manage the exploration/exploitation tradeoff of "EI" and "PI" acquisition function. Defaults to 0.001 for a balance between exploitation and exploration.
n_points (int, optional) – The number of configurations sampled from the search space to infer each batch of new evaluated configurations. Defaults to 10000.
liar_strategy (str, optional) – Definition of the constant value use for the Liar strategy. Can be a value in ["cl_min", "cl_mean", "cl_max"] . Defaults to "cl_max".
n_jobs (int, optional) – Number of parallel processes used to fit the surrogate model of the Bayesian optimization. A value of -1 will use all available cores. Defaults to 1.

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM

class deephyper.search.nas.AgEBO(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, population_size: int = 100, sample_size: int = 10, surrogate_model: str = 'RF', acq_func: str = 'UCB', kappa: float = 0.001, xi: float = 1e-06, n_points: int = 10000, liar_strategy: str = 'cl_max', n_jobs: int = 1, mode: str = 'async', **kwargs)[source]¶

Bases: deephyper.search.nas._regevo.RegularizedEvolution

Aging evolution with Bayesian Optimization.

This algorithm build on the Regularized Evolution. It cumulates Hyperparameter optimization with Bayesian optimisation and Neural architecture search with regularized evolution.

Parameters

problem (NaProblem) – Neural architecture search problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to “.”.
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
population_size (int, optional) – the number of individuals to keep in the population. Defaults to 100.
sample_size (int, optional) – the number of individuals that should participate in each tournament. Defaults to 10.
surrogate_model (str, optional) – Surrogate model used by the Bayesian optimization. Can be a value in ["RF", "ET", "GBRT", "DUMMY"]. Defaults to "RF".
acq_func (str, optional) – Acquisition function used by the Bayesian optimization. Can be a value in ["UCB", "EI", "PI", "gp_hedge"]. Defaults to "UCB".
kappa (float, optional) – Manage the exploration/exploitation tradeoff for the “UCB” acquisition function. Defaults to 0.001 for strong exploitation.
xi (float, optional) – Manage the exploration/exploitation tradeoff of "EI" and "PI" acquisition function. Defaults to 0.000001 for strong exploitation.
n_points (int, optional) – The number of configurations sampled from the search space to infer each batch of new evaluated configurations. Defaults to 10000.
liar_strategy (str, optional) – Definition of the constant value use for the Liar strategy. Can be a value in ["cl_min", "cl_mean", "cl_max"] . Defaults to "cl_max".
n_jobs (int, optional) – Number of parallel processes used to fit the surrogate model of the Bayesian optimization. A value of -1 will use all available cores. Defaults to 1.
mode (str, optional) – Define if the search should be asynchronous or batch synchronous. Choice in [“sync”, “async”]. Defaults to “async”.

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM

class deephyper.search.nas.NeuralArchitectureSearch(problem, evaluator, random_state=None, log_dir='.', verbose=0, **kwargs)[source]¶

Bases: deephyper.search._search.Search

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM

class deephyper.search.nas.Random(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, **kwargs)[source]¶

Bases: deephyper.search.nas._base.NeuralArchitectureSearch

Random neural architecture search. This search algorithm is compatible with a NaProblem defining fixed or variable hyperparameters.

Parameters

problem (NaProblem) – Neural architecture search problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int or RandomState, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to “.”.
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM

class deephyper.search.nas.RegularizedEvolution(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, population_size: int = 100, sample_size: int = 10, **kwargs)[source]¶

Bases: deephyper.search.nas._base.NeuralArchitectureSearch

Regularized evolution neural architecture search. This search is only compatible with a NaProblem that has fixed hyperparameters.

Parameters

problem (NaProblem) – Neural architecture search problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to “.”.
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
population_size (int, optional) – the number of individuals to keep in the population. Defaults to 100.
sample_size (int, optional) – the number of individuals that should participate in each tournament. Defaults to 10.

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM

class deephyper.search.nas.RegularizedEvolutionMixed(problem, evaluator, random_state: Optional[int] = None, log_dir: str = '.', verbose: int = 0, population_size: int = 100, sample_size: int = 10, **kwargs)[source]¶

Bases: deephyper.search.nas._regevo.RegularizedEvolution

Extention of the Regularized evolution neural architecture search to the case of joint hyperparameter and neural architecture search.

Parameters

problem (NaProblem) – Neural architecture search problem describing the search space to explore.
evaluator (Evaluator) – An Evaluator instance responsible of distributing the tasks.
random_state (int, optional) – Random seed. Defaults to None.
log_dir (str, optional) – Log directory where search’s results are saved. Defaults to “.”.
verbose (int, optional) – Indicate the verbosity level of the search. Defaults to 0.
population_size (int, optional) – the number of individuals to keep in the population. Defaults to 100.
sample_size (int, optional) – the number of individuals that should participate in each tournament. Defaults to 10.

search(max_evals: int = - 1, timeout: Optional[int] = None)¶

Execute the search algorithm.

Parameters

max_evals (int, optional) – The maximum number of evaluations of the run function to perform before stopping the search. Defaults to -1, will run indefinitely.
timeout (int, optional) – The time budget of the search before stopping.Defaults to None, will not impose a time budget.

Returns

a pandas DataFrame containing the evaluations performed.

Return type

DataFrame

terminate()¶

Terminate the search.

Raises: SearchTerminationError – raised when the search is terminated with SIGALARM