deephyper.evaluator.storage.Storage#

class deephyper.evaluator.storage.Storage[source]#

Bases: ABC

An abstract interface representing a storage client.

Methods

connect

Connect the storage client to the storage service.

create

Static method allowing the creation of a storage client.

create_new_job

Creates a new job in the store and returns its identifier.

create_new_search

Create a new search in the store and returns its identifier.

load_all_job_ids

Loads the identifiers of all recorded jobs in the search.

load_all_search_ids

Loads the identifiers of all recorded searches.

load_job

Loads the data of a job.

load_jobs

Load all data from a given list of jobs' identifiers.

load_metadata_from_all_jobs

Loads a given metadata value from all jobs.

load_out_from_all_jobs

Loads the output value from all jobs.

load_search

Loads the data of a search.

load_search_value

Loads the value corresponding to key for search_id.

store_job

Stores the value corresponding to key for job_id.

store_job_in

Stores the input arguments of the executed job.

store_job_metadata

Stores other metadata related to the execution of the job.

store_job_out

Stores the output value of the executed job.

store_search_value

Stores the value corresponding to key for search_id.

connect() StorageType[source]#

Connect the storage client to the storage service.

static create(method: str = 'memory', method_kwargs: Dict = None) StorageType[source]#

Static method allowing the creation of a storage client.

Parameters:
  • method (str, optional) – the type of storage client in ["memory", "redis"]. Defaults to “memory”.

  • method_kwargs (Dict, optional) – the client keyword-arguments parameters. Defaults to None.

Raises:

ValueError – if the type of requested storage client is not valid.

Returns:

the created storage client.

Return type:

Storage

abstract create_new_job(search_id: Hashable) Hashable[source]#

Creates a new job in the store and returns its identifier.

Parameters:
  • search_id (Hashable) – The identifier of the search in which a new job

  • created. (is)

Returns:

The created identifier of the job.

Return type:

Hashable

Create a new search in the store and returns its identifier.

Returns:

The identifier of the search.

Return type:

Hashable

abstract load_all_job_ids(search_id: Hashable) List[Hashable][source]#

Loads the identifiers of all recorded jobs in the search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of identifiers of all the jobs.

Return type:

List[Hashable]

abstract load_all_search_ids() List[Hashable][source]#

Loads the identifiers of all recorded searches.

Returns:

A list of identifiers of all the recorded searches.

Return type:

List[Hashable]

abstract load_job(job_id: Hashable) dict[source]#

Loads the data of a job.

Parameters:

job_id (Hashable) – The identifier of the job.

Returns:

The corresponding data of the job.

Return type:

dict

abstract load_jobs(job_ids: List[Hashable]) dict[source]#

Load all data from a given list of jobs’ identifiers.

Parameters:

job_ids (list) – The list of job identifiers.

Returns:

A dictionnary of the retrieved values where the keys are the identifier of jobs.

Return type:

dict

abstract load_metadata_from_all_jobs(search_id: Hashable, key: Hashable) List[Any][source]#

Loads a given metadata value from all jobs.

Parameters:
  • search_id (Hashable) – The identifier of the search.

  • key (Hashable) – The identifier of the value.

Returns:

A list of all the retrieved metadata values.

Return type:

List[Any]

abstract load_out_from_all_jobs(search_id: Hashable) List[Any][source]#

Loads the output value from all jobs.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of all the retrieved output values.

Return type:

List[Any]

Loads the data of a search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

The corresponding data of the search.

Return type:

dict

abstract load_search_value(search_id: Hashable, key: Hashable) Any[source]#

Loads the value corresponding to key for search_id.

Parameters:
  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to access the value.

abstract store_job(job_id: Hashable, key: Hashable, value: Any) None[source]#

Stores the value corresponding to key for job_id.

Parameters:
  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.

abstract store_job_in(job_id: Hashable, args: Tuple = None, kwargs: Dict = None) None[source]#

Stores the input arguments of the executed job.

Parameters:
  • job_id (Hashable) – The identifier of the job.

  • args (Optional[Tuple], optional) – The positional arguments. Defaults to None.

  • kwargs (Optional[Dict], optional) – The keyword arguments. Defaults to None.

abstract store_job_metadata(job_id: Hashable, key: Hashable, value: Any) None[source]#

Stores other metadata related to the execution of the job.

Parameters:
  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the metadata of the given job.

  • value (Any) – The value to store.

abstract store_job_out(job_id: Hashable, value: Any) None[source]#

Stores the output value of the executed job.

Parameters:
  • job_id (Hashable) – The identifier of the job.

  • value (Any) – The value to store.

abstract store_search_value(search_id: Hashable, key: Hashable, value: Any) None[source]#

Stores the value corresponding to key for search_id.

Parameters:
  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.