netunicorn.base.pipeline.CyclePipeline

class CyclePipeline(cycles=None, tasks=(), early_stopping=True, report_results=True, environment_definition=None)

Bases: Pipeline

CyclePipeline is a Pipeline that will be executed several times. All defined stages would be executed and then the execution would continue from the first stage again.

Parameters:

• cycles (Optional[int], default: None)

• tasks (Collection[Union[Task, TaskDispatcher, Collection[Union[Task, TaskDispatcher]]]], default: ())

• early_stopping (bool, default: True)

• report_results (bool, default: True)

• environment_definition (Optional[EnvironmentDefinition], default: None)

Attributes

last_stage	Current last stage of the pipeline.
name	Execution Graph name.
early_stopping	Whether to stop executing tasks after a first failure.
report_results	Whether executor should connect core services to report execution results in the end.
environment_definition	Environment definition for the execution graph.
graph	Graph of tasks and their order.
override_graph_validation	Disable graph validation.

Methods

copy	Return a copy of the pipeline.
draw	Draw execution graph using networkx library.
is_execution_graph_valid	Validates execution graph according to the ExecutionGraph rules.
then	Add a task or list of tasks as a separate stage to the end of the pipeline.

copy()

Return a copy of the pipeline.

Returns:

Pipeline – a copy of the pipeline

draw(nx_layout_function=<function shell_layout>)

Draw execution graph using networkx library.

Parameters:

nx_layout_function (default: <function shell_layout at 0x7fdfbd01c280>) – networkx layout function to use for drawing (from networkx.drawing.layout)

Returns:

None – None

static is_execution_graph_valid(obj)

Validates execution graph according to the ExecutionGraph rules.

Returns:

bool – True if execution graph is valid, raises an exception otherwise

Parameters:

obj (ExecutionGraph)

then(element)

Add a task or list of tasks as a separate stage to the end of the pipeline.

Parameters:

element (Union[Task, TaskDispatcher, Collection[Union[Task, TaskDispatcher]]]) – a task or tuple of tasks to be added

Returns:

Pipeline – self

last_stage: Union[str, int]

Current last stage of the pipeline.

name: str

Execution Graph name.

early_stopping: bool

Whether to stop executing tasks after a first failure.

report_results: bool

Whether executor should connect core services to report execution results in the end.

environment_definition: EnvironmentDefinition

Environment definition for the execution graph.

graph

Graph of tasks and their order.

override_graph_validation

Disable graph validation. Executor and other components will not validate the graph before execution.