parsl.dataflow.memoization.Memoizer

class parsl.dataflow.memoization.Memoizer

Defines the interface for the DFK to talk to the memoization/checkpoint system.

The DFK will invoke these methods on an instance of a Memoizer at suitable points in the lifecycle of a task. These methods are not intended for users to invoke directly.

__init__()

Methods

__init__()
check_memo(task)	Asks the checkpoint system for a result recorded for the described task.
close()	Called at DFK shutdown.
start(*, run_dir, config_run_dir)	Called by the DFK when it starts up.
update_memo_exception(task, e)	Called by the DFK when a task completes with an exception.
update_memo_result(task, r)	Called by the DFK when a task completes with a successful result.

abstractmethod check_memo(task: TaskRecord) → Future[Any] | None

Asks the checkpoint system for a result recorded for the described task. check_memo should return a Future that will be used as an executor future, in place of sending the task to an executor for execution. That future should be populated with a result or exception.

abstractmethod close() → None

Called at DFK shutdown. This gives the checkpoint system an opportunity for graceful shutdown.

abstractmethod start(*, run_dir: str, config_run_dir: str) → None

Called by the DFK when it starts up.

This is an opportunity for the memoization/checkpoint system to initialize itself.

The path to the per-run run directory and the base run directory are passed as parameters.

abstractmethod update_memo_exception(task: TaskRecord, e: BaseException) → None

Called by the DFK when a task completes with an exception.

On every task completion, either this method or update_memo_result will be called, but not both.

This is an opportunity for the memoization/checkpoint system to record the outcome of a task for later discovery by a call to check_memo.

abstractmethod update_memo_result(task: TaskRecord, r: Any) → None

Called by the DFK when a task completes with a successful result.

On every task completion, either this method or update_memo_exception will be called, but not both.

This is an opportunity for the memoization/checkpoint system to record the outcome of a task for later discovery by a call to check_memo.