parsl.providers.base.ExecutionProvider

class parsl.providers.base.ExecutionProvider

Execution providers are responsible for managing execution resources that have a Local Resource Manager (LRM). For instance, campus clusters and supercomputers generally have LRMs (schedulers) such as Slurm, Torque/PBS, and Condor. Clouds, on the other hand, have API interfaces that allow much more fine-grained composition of an execution environment. An execution provider abstracts these types of resources and provides a single uniform interface to them.

The providers abstract away the interfaces provided by various systems to request, monitor, and cancel compute resources.

                      +------------------
                      |
script_string ------->|  submit
     id      <--------|---+
                      |
[ ids ]       ------->|  status
[statuses]   <--------|----+
                      |
[ ids ]       ------->|  cancel
[cancel]     <--------|----+
                      |
                      +-------------------

In addition to the listed methods, an ExecutionProvider instance must always have these attributes, which both default to None:

mem_per_node: Real memory to provision per node in GB.

Providers which set this attribute should ask for mem_per_node of memory when provisioning resources, and set the corresponding environment variable PARSL_MEMORY_GB before executing submitted commands.

If this attribute is set, executors may use it to calculate how many tasks can run concurrently per node.

cores_per_node: Number of cores to provision per node.

Providers which set this attribute should ask for cores_per_node cores when provisioning resources, and set the corresponding environment variable PARSL_CORES before executing submitted commands.

If this attribute is set, executors may use it to calculate how many tasks can run concurrently per node.

abstractmethod __init__() → None

Methods

__init__()
cancel(job_ids)	Cancels the resources identified by the job_ids provided by the user.
status(job_ids)	Get the status of a list of jobs identified by the job identifiers returned from the submit request.
submit(command, tasks_per_node[, job_name])	The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as for HighThroughputExecutor or WorkQueueExecutor).

Attributes

label	Provides the label for this provider
status_polling_interval	Returns the interval, in seconds, at which the status method should be called.

abstractmethod cancel(job_ids: List[object]) → List[bool]

Cancels the resources identified by the job_ids provided by the user.

Parameters:

job_ids (-) – A list of job identifiers

Returns:

• A list of status from cancelling the job which can be True, False

Raises:

- ExecutionProviderException or its subclasses –

abstract property label: str

Provides the label for this provider

abstractmethod status(job_ids: List[object]) → List[JobStatus]

Get the status of a list of jobs identified by the job identifiers returned from the submit request.

Parameters:

job_ids (-) – A list of job identifiers

Returns:

• A list of JobStatus objects corresponding to each job_id in the job_ids list.

Raises:

- ExecutionProviderException or its subclasses –

abstract property status_polling_interval: int

Returns the interval, in seconds, at which the status method should be called.

Returns:

the number of seconds to wait between calls to status()

abstractmethod submit(command: str, tasks_per_node: int, job_name: str = 'parsl.auto') → object

The submit method takes the command string to be executed upon instantiation of a resource most often to start a pilot (such as for HighThroughputExecutor or WorkQueueExecutor).

Args :

• command (str) : The bash command string to be executed

• tasks_per_node (int) : command invocations to be launched per node

KWargs:

• job_name (str) : Human friendly name to be assigned to the job request

Returns:

• A job identifier, this could be an integer, string etc or None or any other object that evaluates to boolean false if submission failed but an exception isn’t thrown.

Raises:

- ExecutionProviderException or its subclasses –