Pipelines

In parameterisation is described how classes are created, which declare the parameters and link calculations or visualisations. In this section you will learn how you can connect several such panels with a pipeline to express complex workflows in which the output of one stage is fed into the next stage.

import param
import panel as pn

pn.extension('katex')

pipeline = pn.pipeline.Pipeline()

While we saw earlier how methods are linked to the param.depends decorator, pipelines use a different decorator and a convention for displaying the objects. The param.output decorator provides a way to annotate the methods of a class by declaring its output. Pipeline uses this information to determine what outputs are available to be fed into the next stage of the workflow. In the following example, the class Stage1 has two parameters (a and b) and an output c. The decorator’s signature allows a number of different ways to declare the outputs:

• param.output(): If output is declared with no arguments, the method returns output that inherits the name of the method and does not make any specific type declarations.

• param.output(param.Number): When declaring an output with a specific parameter or a Python type, the output is declared with a specific type.

• param.output(c=param.Number): If an output is declared with a keyword argument, you can overwrite the method name as the name of the output and declare the type.

It is also possible to declare several parameters as keywords or as tuples:

• param.output(c=param.Number, d=param.String)

• param.output(('c', param.Number), ('d', param.String))

In the example below, the output is simply the result of multiplying the two inputs (a and b) that produce the output c. In addition, we declare a view method that returns a LaTeX pane. Finally, a panel method returns a Panel object that render both the parameters and the view.

class Stage1(param.Parameterized):

    a = param.Number(default=5, bounds=(0, 10))

    b = param.Number(default=5, bounds=(0, 10))

    @param.output(('c', param.Number), ('d', param.Number))
    def output(self):
        return self.a * self.b, self.a ** self.b

    @param.depends('a', 'b')
    def view(self):
        c, d = self.output()
        return pn.pane.LaTeX('${a} * {b} = {c}$\n${a}^{{{b}}} = {d}$'.format(
            a=self.a, b=self.b, c=c, d=d))

    def panel(self):
        return pn.Row(self.param, self.view)

stage1 = Stage1()
stage1.panel()

In summary, we followed a few conventions to create this stage of our pipeline:

1. Declare a parameterised class with some input parameters

2. Declare and name one or more output methods

3. Declare a panel method that returns a View of the object that the pipeline can render.

Now that the object has been instantiated, we can also ask it about its outputs:

stage1.param.outputs()

{'c': (<param.Number at 0x11ab652c0>,
  <bound method Stage1.output of Stage1(a=5, b=5, name='Stage101510')>,
  0),
 'd': (<param.Number at 0x11ab65530>,
  <bound method Stage1.output of Stage1(a=5, b=5, name='Stage101510')>,
  1)}

We can see that Stage1 declared an output with the name c of the type Number that can be accessed using the output method. Now let’s add stage1 with add_stage to our pipeline:

pipeline.add_stage('Stage 1', stage1)

For a pipeline, however, we still need at least one stage2 that processes the result of stage1. Therefore a parameter c should be declared from the result of stage1. As a further parameter, we define exp and a view method again, which depends on the two parameters and the panel method.

class Stage2(param.Parameterized):

    c = param.Number(default=5, precedence=-1, bounds=(0, None))

    exp = param.Number(default=0.1, bounds=(0, 3))

    @param.depends('c', 'exp')
    def view(self):
        return pn.pane.LaTeX('${%s}^{%s}={%.3f}$' % (self.c, self.exp, self.c**self.exp))

    def panel(self):
        return pn.Row(self.param, self.view)

stage2 = Stage2(c=stage1.output()[0])
stage2.panel()

Also, we now add stage2 to the pipeline object:

pipeline.add_stage('Stage 2', stage2)

We now have a two-stage pipeline where the output c is passed from stage1 to stage2. Now we can display the pipeline with pipeline.layout:

pipeline.layout

The rendering of the pipeline shows a small diagram with the available workflow stages and the Previous and Next buttons to switch between the individual phases. This enables navigation even in more complex workflows with many more phases.

Above we instantiated each level individually. However, if the pipeline is to be deployed as a server app, the stages can also be declared as part of the constructor:

stages = [
    ('Stage 1', Stage1),
    ('Stage 2', Stage2)
]

pipeline = pn.pipeline.Pipeline(stages)
pipeline.layout

The pipeline stages can either be Parameterized instances or Parameterized classes. With instances, however, you have to make sure that the update of the parameters of the class also updates the current status of the class.