pytams.taskrunner

Exceptions

RunnerError

Exception class for the runner.

Classes

WorkerLoggerPlugin	A plugin to configure logging on each worker.
BaseRunner	An ABC for the task runners.
AsIORunner	A task runner class based on asyncIO.
DaskRunner	A task runner class based on Dask.

Functions

get_runner_type(→ type[BaseRunner])

Create an engine from parameters.

Module Contents

class WorkerLoggerPlugin(params: dict[Any, Any])

Bases: dask.distributed.WorkerPlugin

A plugin to configure logging on each worker.

setup(worker: dask.distributed.Worker) → None

Configure logging on the worker.

Parameters:

worker – the dask worker

exception RunnerError

Bases: Exception

Exception class for the runner.

class BaseRunner(params: dict, sync_wk: collections.abc.Callable, n_workers: int = 1)

An ABC for the task runners.

abstract make_promise(task: list[Any]) → None

Log a new task to the list of task to tackle.

abstract execute_promises() → Any

Execute the list of promises.

abstract n_workers() → int

Return the number of workers in the runner.

class AsIORunner(params: dict, sync_wk: collections.abc.Callable, n_workers: int = 1)

Bases: BaseRunner

A task runner class based on asyncIO.

An runner that relies on asyncio to schedule a tasks concurently in worker processes. Tasks are added to an internal queue from which worker can take them and put the results back into result queue.

async add_task(task: list[Any]) → None

Append a task to the queue.

make_promise(task: list[Any]) → None

A synchronous wrapper to add_task.

async run_tasks() → list[Any]

Create worker tasks and run.

execute_promises() → Any

A synchronous wrapper to run_tasks.

n_workers() → int

Return the number of workers in the runner.

class DaskRunner(params: dict, sync_wk: collections.abc.Callable, n_workers: int = 1)

Bases: BaseRunner

A task runner class based on Dask.

An runner that relies on dask to schedule a tasks concurently in workers.

dask_backend

make_promise(task: list[Any]) → None

Append a task to the internal task list.

just_delay(obj: Any) → Any

Delay an object.

execute_promises() → Any

Execute a list of promises.

Parameters:

list_of_p – a list of dask promises

Returns:

A list with the return argument of each promised task.

Raises:

Exception if compute fails (raise internal error) –

n_workers() → int

Return the number of workers in the runner.

get_runner_type(params: dict) → type[BaseRunner]

Create an engine from parameters.