deephyper.evaluator.storage.MemoryStorage#

class deephyper.evaluator.storage.MemoryStorage[source]#

Bases: Storage

Storage client for local in-memory storage.

This backend does not allow to share the data between evaluators running in different processes.

Methods

connect

Connect the storage client to the storage service.

create

Static method allowing the creation of a storage client.

create_new_job

Creates a new job in the store and returns its identifier.

create_new_search

Create a new search in the store and returns its identifier.

load_all_job_ids

Loads the identifiers of all recorded jobs in the search.

load_all_search_ids

Loads the identifiers of all recorded searches.

load_job

Loads the data of a job.

load_jobs

Load all data from a given list of jobs' identifiers.

load_metadata_from_all_jobs

Loads a given metadata value from all jobs.

load_out_from_all_jobs

Loads the output value from all jobs.

load_search

Loads the data of a search.

load_search_value

Loads the value corresponding to key for search_id.

store_job

Stores the value corresponding to key for job_id.

store_job_in

Stores the input arguments of the executed job.

store_job_metadata

Stores other metadata related to the execution of the job.

store_job_out

Stores the output value of the executed job.

store_search_value

Stores the value corresponding to key for search_id.
connect() → StorageType#

Connect the storage client to the storage service.

static create(method: str = 'memory', method_kwargs: Dict = None) → StorageType#

Static method allowing the creation of a storage client.

Parameters:

  • method (str, optional) – the type of storage client in ["memory", "redis"]. Defaults to “memory”.

  • method_kwargs (Dict, optional) – the client keyword-arguments parameters. Defaults to None.

Raises:

ValueError – if the type of requested storage client is not valid.

Returns:

the created storage client.

Return type:

Storage

create_new_job(search_id: Hashable) → Hashable[source]#

Creates a new job in the store and returns its identifier.

Parameters:

  • search_id (Hashable) – The identifier of the search in which a new job

  • created. (is) –

Returns:

The created identifier of the job.

Return type:

Hashable

Create a new search in the store and returns its identifier.

Returns:

The identifier of the search.

Return type:

Hashable

load_all_job_ids(search_id: Hashable) → List[Hashable][source]#

Loads the identifiers of all recorded jobs in the search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of identifiers of all the jobs.

Return type:

List[Hashable]

load_all_search_ids() → List[Hashable][source]#

Loads the identifiers of all recorded searches.

Returns:

A list of identifiers of all the recorded searches.

Return type:

List[Hashable]

load_job(job_id: Hashable) → dict[source]#

Loads the data of a job.

Parameters:

job_id (Hashable) – The identifier of the job.

Returns:

The corresponding data of the job.

Return type:

dict

load_jobs(job_ids: List[Hashable]) → dict[source]#

Load all data from a given list of jobs’ identifiers.

Parameters:

job_ids (list) – The list of job identifiers.

Returns:

A dictionnary of the retrieved values where the keys are the identifier of jobs.

Return type:

dict

load_metadata_from_all_jobs(search_id: Hashable, key: Hashable) → List[Any][source]#

Loads a given metadata value from all jobs.

Parameters:

  • search_id (Hashable) – The identifier of the search.

  • key (Hashable) – The identifier of the value.

Returns:

A list of all the retrieved metadata values.

Return type:

List[Any]

load_out_from_all_jobs(search_id: Hashable) → List[Any][source]#

Loads the output value from all jobs.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of all the retrieved output values.

Return type:

List[Any]

Loads the data of a search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

The corresponding data of the search.

Return type:

dict

load_search_value(search_id: Hashable, key: Hashable) → Any[source]#

Loads the value corresponding to key for search_id.

Parameters:

  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to access the value.

store_job(job_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores the value corresponding to key for job_id.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.

store_job_in(job_id: Hashable, args: Tuple = None, kwargs: Dict = None) → None[source]#

Stores the input arguments of the executed job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • args (Optional[Tuple], optional) – The positional arguments. Defaults to None.

  • kwargs (Optional[Dict], optional) – The keyword arguments. Defaults to None.

store_job_metadata(job_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores other metadata related to the execution of the job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the metadata of the given job.

  • value (Any) – The value to store.

store_job_out(job_id: Hashable, value: Any) → None[source]#

Stores the output value of the executed job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • value (Any) – The value to store.

store_search_value(search_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores the value corresponding to key for search_id.

Parameters:

  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.