deephyper.problem.NaProblem#
- class deephyper.problem.NaProblem[source]#
Bases:
object
A Neural Architecture Problem specification for Neural Architecture Search.
>>> from deephyper.problem import NaProblem >>> from deephyper.nas.preprocessing import minmaxstdscaler >>> from deepspace.tabular import OneLayerSpace >>> Problem = NaProblem() >>> Problem.load_data(load_data) >>> Problem.preprocessing(minmaxstdscaler) >>> Problem.search_space(OneLayerSpace) >>> Problem.hyperparameters( ... batch_size=100, ... learning_rate=0.1, ... optimizer='adam', ... num_epochs=10, ... callbacks=dict( ... EarlyStopping=dict( ... monitor='val_r2', ... mode='max', ... verbose=0, ... patience=5 ... ) ... ) ... ) >>> Problem.loss('mse') >>> Problem.metrics(['r2']) >>> Problem.objective('val_r2__last')
Methods
Add hyperparameters to search the neural architecture search problem.
augment
- meta private:
Build and return a search space object using the infered data shapes after loading data.
check_objective
- meta private:
Extract the value of hyperparameters present in
config
based on the defined hyperparameters in the currentNaProblem
Generate a
dict
configuration from thearch_seq
andhp_values
passed.Get a keras model object from a set of decisions in the current search space.
Define hyperparameters used to evaluate generated architectures.
Define the function loading the data.
Define the loss used to train generated architectures.
Define a list of metrics for the training of generated architectures.
Define the objective you want to maximize for the search.
Define how to preprocess your data.
Set a search space for neural architecture search.
Attributes
The default configuration as a dictionnary.
The list of hyperparameters names.
space
- add_hyperparameter(value, name: str | None = None, default_value=None) Hyperparameter [source]#
Add hyperparameters to search the neural architecture search problem.
>>> Problem.hyperparameters( ... batch_size=problem.add_hyperparameter((32, 256), "batch_size") ... )
- Parameters:
value – a hyperparameter description.
name – a name of the defined hyperparameter, the same as the current key.
default_value (Optional) – a default value of the hyperparameter.
- Returns:
the defined hyperparameter.
- Return type:
Hyperparameter
- build_search_space(seed=None)[source]#
Build and return a search space object using the infered data shapes after loading data.
- Returns:
A search space instance.
- Return type:
- property default_hp_configuration#
The default configuration as a dictionnary.
- extract_hp_values(config)[source]#
Extract the value of hyperparameters present in
config
based on the defined hyperparameters in the currentNaProblem
- gen_config(arch_seq: list, hp_values: list) dict [source]#
Generate a
dict
configuration from thearch_seq
andhp_values
passed.
- get_keras_model(arch_seq: list) tensorflow.keras.Model [source]#
Get a keras model object from a set of decisions in the current search space. :param arch_seq: a list of int of floats describing a choice of operations for the search space defined in the current problem. :type arch_seq: list
- property hyperparameter_names#
The list of hyperparameters names.
- hyperparameters(**kwargs)[source]#
Define hyperparameters used to evaluate generated architectures.
Hyperparameters can be defined such as:
Problem.hyperparameters( batch_size=256, learning_rate=0.01, optimizer="adam", num_epochs=20, verbose=0, callbacks=dict(...), )
- load_data(func: callable, **kwargs)[source]#
Define the function loading the data. .. code-block:: python
Problem.load_data(load_data, load_data_kwargs)
This
load_data
callable can follow two different interfaces: Numpy arrays or generators.Numpy arrays:
In the case of Numpy arrays, the callable passed to
Problem.load_data(...)
has to return the following tuple:(X_train, y_train), (X_valid, y_valid)
. In the most simple case where the model takes a single input, each of these elements is a Numpy array. Generally,X_train
andy_train
have to be of the same length (i.e., samearray.shape[0]
) which is also the case forX_valid
andy_valid
. Similarly, the shape of the elements ofX_train
andX_valid
which is also the case fory_train
andy_valid
. An exampleload_data
function can beimport numpy as np def load_data(N=100): X = np.zeros((N, 1)) y = np.zeros((N,1)) return (X, y), (X, y)
It is also possible for the model to take several inputs. In fact, experimentaly it can be notices that separating some inputs with different inputs can significantly help the learning of the model. Also, sometimes different inputs may be of the “types” for example two molecular fingerprints. In this case, it can be very interesting to share the weights of the model to process these two inputs. In the case of multi-inputs models the
load_data
function will also return(X_train, y_train), (X_valid, y_valid)
bu whereX_train
andX_valid
are two lists of Numpy arrays. For example, the following is correct:import numpy as np def load_data(N=100): X = np.zeros((N, 1)) y = np.zeros((N,1)) return ([X, X], y), ([X, X], y)
Generators:
Returning generators with a single input:
def load_data(N=100): tX, ty = np.zeros((N,1)), np.zeros((N,1)) vX, vy = np.zeros((N,1)), np.zeros((N,1)) def train_gen(): for x, y in zip(tX, ty): yield ({"input_0": x}, y) def valid_gen(): for x, y in zip(vX, vy): yield ({"input_0": x}, y) res = { "train_gen": train_gen, "train_size": N, "valid_gen": valid_gen, "valid_size": N, "types": ({"input_0": tf.float64}, tf.float64), "shapes": ({"input_0": (1, )}, (1, )) } return res
Returning generators with multiple inputs:
def load_data(N=100): tX0, tX1, ty = np.zeros((N,1)), np.zeros((N,1)), np.zeros((N,1)), vX0, vX1, vy = np.zeros((N,1)), np.zeros((N,1)), np.zeros((N,1)), def train_gen(): for x0, x1, y in zip(tX0, tX1, ty): yield ({ "input_0": x0, "input_1": x1 }, y) def valid_gen(): for x0, x1, y in zip(vX0, vX1, vy): yield ({ "input_0": x0, "input_1": x1 }, y) res = { "train_gen": train_gen, "train_size": N, "valid_gen": valid_gen, "valid_size": N, "types": ({"input_0": tf.float64, "input_1": tf.float64}, tf.float64), "shapes": ({"input_0": (5, ), "input_1": (5, )}, (1, )) } return res
- Parameters:
func (callable) – the load data function.
- loss(loss, loss_weights=None, class_weights=None)[source]#
Define the loss used to train generated architectures.
It can be a
str
corresponding to a Keras loss function:problem.loss("categorical_crossentropy")
A custom loss function can also be defined:
def NLL(y, rv_y): return -rv_y.log_prob(y) problem.loss(NLL)
The loss can be automatically searched:
problem.loss( problem.add_hyperparameter( ["mae", "mse", "huber_loss", "log_cosh", "mape", "msle"], "loss" ) )
It is possible to define a different loss for each output:
problem.loss( loss={"output_0": "mse", "output_1": "mse"}, loss_weights={"output_0": 0.0, "output_1": 1.0}, )
- metrics(metrics=None)[source]#
Define a list of metrics for the training of generated architectures.
A list of metrics can be defined to be monitored or used as an objective. It can be a keyword or a callable. For example, if it is a keyword:
problem.metrics(["acc"])
In case you need multiple metrics:
problem.metrics["mae", "mse"]
In case you want to use a custom metric:
def sparse_perplexity(y_true, y_pred): cross_entropy = tf.keras.losses.sparse_categorical_crossentropy(y_true, y_pred) perplexity = tf.pow(2.0, cross_entropy) return perplexity problem.metrics([sparse_perplexity])
- objective(objective)[source]#
Define the objective you want to maximize for the search.
If you want to use the validation accuracy at the last epoch:
problem.objective("val_acc")
Note
Be sure to define
acc
in theproblem.metrics(["acc"])
.It can accept some prefix and suffix such as
__min, __max, __last
:problem.objective("-val_acc__max")
It can be a
callable
:def myobjective(history: dict) -> float: return history["val_acc"][-1] problem.objective(myobjective)
- Parameters:
objective (str or callable) – The objective will be maximized. If
objective
isstr
then it should be either ‘loss’ or a defined metric. You can use the'val_'
prefix when you want to select the objective on the validation set. You can use one of['__min', '__max', '__last']
which respectively means you want to select the min, max or last value among all epochs. Using ‘__last’ will save a consequent compute time because the evaluation will not compute metrics on validation set for all epochs but the last. Ifobjective
is callable it should return a scalar value (i.e. float) and it will take adict
parameter. Thedict
will contain keys corresponding to loss and metrics such as['loss', 'val_loss', 'r2', 'val_r2]
, it will also contains'n_parameters'
which corresponds to the number of trainable parameters of the current model,'training_time'
which corresponds to the time required to train the model,'predict_time'
which corresponds to the time required to make a prediction over the whole validation set. If this callable has a'__last'
suffix then the evaluation will only compute validation loss/metrics for the last epoch. If this callable has contains ‘with_pred’ in its name then thedict
will have two other keys['y_pred', 'y_true']
where'y_pred
corresponds to prediction of the model on validation set and'y_true'
corresponds to real prediction.- Raises:
WrongProblemObjective – raised when the objective is of a wrong definition.
- preprocessing(func: callable)[source]#
Define how to preprocess your data.
- Parameters:
func (callable) – a function which returns a preprocessing scikit-learn
Pipeline
.
- search_space(space_class, **kwargs)[source]#
Set a search space for neural architecture search.
- Parameters:
space_class (KSearchSpace) – an object of type
KSearchSpace
which has to implement thebuild()
method.- Raises:
SearchSpaceBuilderMissingParameter – raised when either of
(input_shape, output_shape)
are missing parameters offunc
.