deephyper.evaluator.storage.RayStorage#

class deephyper.evaluator.storage.RayStorage(address='auto')[source]#

Bases: Storage

Storage class using Ray actors. The RayStorage is wrapping the MemoryStorage class to be a Ray actor.

Parameters:

address (str, optional) – address of the Ray-head. Defaults to "auto", to connect to the local head node.

Methods

connect

Connect the storage client to the storage service.

create

Static method allowing the creation of a storage client.

create_new_job

Creates a new job in the store and returns its identifier.

create_new_search

Create a new search in the store and returns its identifier.

load_all_job_ids

Loads the identifiers of all recorded jobs in the search.

load_all_search_ids

Loads the identifiers of all recorded searches.

load_job

Loads the data of a job.

load_jobs

Load all data from a given list of jobs' identifiers.

load_metadata_from_all_jobs

Loads a given metadata value from all jobs.

load_out_from_all_jobs

Loads the output value from all jobs.

load_search

Loads the data of a search.

load_search_value

Loads the value corresponding to key for search_id.

store_job

Stores the value corresponding to key for job_id.

store_job_in

Stores the input arguments of the executed job.

store_job_metadata

Stores other metadata related to the execution of the job.

store_job_out

Stores the output value of the executed job.

store_search_value

Stores the value corresponding to key for search_id.

Attributes

ray_storage_counter

connect() → StorageType#

Connect the storage client to the storage service.

static create(method: str = 'memory', method_kwargs: Dict = None) → StorageType#

Static method allowing the creation of a storage client.

Parameters:

  • method (str, optional) – the type of storage client in ["memory", "redis"]. Defaults to “memory”.

  • method_kwargs (Dict, optional) – the client keyword-arguments parameters. Defaults to None.

Raises:

ValueError – if the type of requested storage client is not valid.

Returns:

the created storage client.

Return type:

Storage

create_new_job(search_id: Hashable) → Hashable[source]#

Creates a new job in the store and returns its identifier.

Parameters:

  • search_id (Hashable) – The identifier of the search in which a new job

  • created. (is) –

Returns:

The created identifier of the job.

Return type:

Hashable

Create a new search in the store and returns its identifier.

Returns:

The identifier of the search.

Return type:

Hashable

load_all_job_ids(search_id: Hashable) → List[Hashable][source]#

Loads the identifiers of all recorded jobs in the search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of identifiers of all the jobs.

Return type:

List[Hashable]

load_all_search_ids() → List[Hashable][source]#

Loads the identifiers of all recorded searches.

Returns:

A list of identifiers of all the recorded searches.

Return type:

List[Hashable]

load_job(job_id: Hashable) → dict[source]#

Loads the data of a job.

Parameters:

job_id (Hashable) – The identifier of the job.

Returns:

The corresponding data of the job.

Return type:

dict

load_jobs(job_ids: List[Hashable]) → dict[source]#

Load all data from a given list of jobs’ identifiers.

Parameters:

job_ids (list) – The list of job identifiers.

Returns:

A dictionnary of the retrieved values where the keys are the identifier of jobs.

Return type:

dict

load_metadata_from_all_jobs(search_id: Hashable, key: Hashable) → List[Any][source]#

Loads a given metadata value from all jobs.

Parameters:

  • search_id (Hashable) – The identifier of the search.

  • key (Hashable) – The identifier of the value.

Returns:

A list of all the retrieved metadata values.

Return type:

List[Any]

load_out_from_all_jobs(search_id: Hashable) → List[Any][source]#

Loads the output value from all jobs.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

A list of all the retrieved output values.

Return type:

List[Any]

Loads the data of a search.

Parameters:

search_id (Hashable) – The identifier of the search.

Returns:

The corresponding data of the search.

Return type:

dict

load_search_value(search_id: Hashable, key: Hashable) → Any[source]#

Loads the value corresponding to key for search_id.

Parameters:

  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to access the value.

store_job(job_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores the value corresponding to key for job_id.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.

store_job_in(job_id: Hashable, args: Tuple = None, kwargs: Dict = None) → None[source]#

Stores the input arguments of the executed job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • args (Optional[Tuple], optional) – The positional arguments. Defaults to None.

  • kwargs (Optional[Dict], optional) – The keyword arguments. Defaults to None.

store_job_metadata(job_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores other metadata related to the execution of the job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the metadata of the given job.

  • value (Any) – The value to store.

store_job_out(job_id: Hashable, value: Any) → None[source]#

Stores the output value of the executed job.

Parameters:

  • job_id (Hashable) – The identifier of the job.

  • value (Any) – The value to store.

store_search_value(search_id: Hashable, key: Hashable, value: Any) → None[source]#

Stores the value corresponding to key for search_id.

Parameters:

  • search_id (Hashable) – The identifier of the job.

  • key (Hashable) – A key to use to store the value.

  • value (Any) – The value to store.