Processors

Processors are plugins that can execute code on certain events. For more information on those events check the class documentation.

You can see processors as an alternative way of dealing with the results of a task, however, it has a few advantages:

  1. Due to its event-based nature, you can process the events asynchronously. This means that you will be processing the result of a host exactly once the host is completed without the need to wait for the rest of the hosts to complete.

  2. Because you are tapping into events code is more concise and easier to understand.

Let’s see how processors work with a few examples. Let’s start by loading the nornir object and some libraries we will need:

  1. [1]:
  1. from typing import Dict
  3. from nornir import InitNornir
  5. nr = InitNornir(config_file="config.yaml")

Now let’s write a processor that will print on screen some information about the execution of the task:

  1. [2]:
  1. # note that these imports are only needed if you are annotating your code with types
  2. from typing import Dict
  4. from nornir.core import Nornir
  5. from nornir.core.inventory import Host
  6. from nornir.core.task import AggregatedResult, MultiResult, Result, Task
  8. class PrintResult:
  9.     def task_started(self, task: Task) -> None:
  10.         print(f">>> starting: {task.name}")
  12.     def task_completed(self, task: Task, result: AggregatedResult) -> None:
  13.         print(f">>> completed: {task.name}")
  15.     def task_instance_started(self, task: Task, host: Host) -> None:
  16.         pass
  18.     def task_instance_completed(
  19.         self, task: Task, host: Host, result: MultiResult
  20.     ) -> None:
  21.         print(f"  - {host.name}: - {result.result}")
  23.     def subtask_instance_started(self, task: Task, host: Host) -> None:
  24.         pass  # to keep example short and sweet we ignore subtasks
  26.     def subtask_instance_completed(
  27.         self, task: Task, host: Host, result: MultiResult
  28.     ) -> None:
  29.         pass  # to keep example short and sweet we ignore subtasks

Now we are going to write another processor that will save some information about the task in a dictionary:

  1. [3]:
  1. class SaveResultToDict:
  2.     def __init__(self, data: Dict[str, None]) -> None:
  3.         self.data = data
  5.     def task_started(self, task: Task) -> None:
  6.         self.data[task.name] = {}
  7.         self.data[task.name]["started"] = True
  9.     def task_completed(self, task: Task, result: AggregatedResult) -> None:
  10.         self.data[task.name]["completed"] = True
  12.     def task_instance_started(self, task: Task, host: Host) -> None:
  13.         self.data[task.name][host.name] = {"started": True}
  15.     def task_instance_completed(
  16.         self, task: Task, host: Host, result: MultiResult
  17.     ) -> None:
  18.         self.data[task.name][host.name] = {
  19.             "completed": True,
  20.             "result": result.result,
  21.         }
  23.     def subtask_instance_started(self, task: Task, host: Host) -> None:
  24.         pass  # to keep example short and sweet we ignore subtasks
  26.     def subtask_instance_completed(
  27.         self, task: Task, host: Host, result: MultiResult
  28.     ) -> None:
  29.         pass  # to keep example short and sweet we ignore subtasks

Finally, to test the processors we are going to use a very simple task that will just greet us on behalf of each device:

  1. [4]:
  1. def greeter(task: Task, greet: str) -> Result:
  2.     return Result(host=task.host, result=f"{greet}! my name is {task.host.name}")

Hopefully everything is clear so far, let’s now put it to use:

  1. [5]:
  1. # NBVAL_IGNORE_OUTPUT
  3. data = {}  # this is the dictionary where SaveResultToDict will store the information
  5. # similary to .filter, with_processors returns a copy of the nornir object but with
  6. # the processors assigned to it. Let's now use the method to assign both processors
  7. nr_with_processors = nr.with_processors([SaveResultToDict(data), PrintResult()])
  9. # now we can use nr_with_processors to execute our greeter task
  10. nr_with_processors.run(
  11.     name="hi!",
  12.     task=greeter,
  13.     greet="hi",
  14. )
  15. nr_with_processors.run(
  16.     name="bye!",
  17.     task=greeter,
  18.     greet="bye",
  19. )
  1. >>> starting: hi!
  2.   - host1.cmh: - hi! my name is host1.cmh
  3.   - host2.cmh: - hi! my name is host2.cmh
  4.   - spine00.cmh: - hi! my name is spine00.cmh
  5.   - spine01.cmh: - hi! my name is spine01.cmh
  6.   - leaf00.cmh: - hi! my name is leaf00.cmh
  7.   - leaf01.cmh: - hi! my name is leaf01.cmh
  8.   - host1.bma: - hi! my name is host1.bma
  9.   - host2.bma: - hi! my name is host2.bma
  10.   - spine00.bma: - hi! my name is spine00.bma
  11.   - spine01.bma: - hi! my name is spine01.bma
  12.   - leaf00.bma: - hi! my name is leaf00.bma  - leaf01.bma: - hi! my name is leaf01.bma
  14. >>> completed: hi!
  15. >>> starting: bye!
  16.   - host1.cmh: - bye! my name is host1.cmh
  17.   - host2.cmh: - bye! my name is host2.cmh
  18.   - spine00.cmh: - bye! my name is spine00.cmh
  19.   - spine01.cmh: - bye! my name is spine01.cmh
  20.   - leaf00.cmh: - bye! my name is leaf00.cmh
  21.   - leaf01.cmh: - bye! my name is leaf01.cmh
  22.   - host1.bma: - bye! my name is host1.bma
  23.   - host2.bma: - bye! my name is host2.bma
  24.   - spine00.bma: - bye! my name is spine00.bma
  25.   - spine01.bma: - bye! my name is spine01.bma
  26.   - leaf00.bma: - bye! my name is leaf00.bma  - leaf01.bma: - bye! my name is leaf01.bma
  28. >>> completed: bye!
  1. [5]:
  1. AggregatedResult (bye!): {'host1.cmh': MultiResult: [Result: "bye!"], 'host2.cmh': MultiResult: [Result: "bye!"], 'spine00.cmh': MultiResult: [Result: "bye!"], 'spine01.cmh': MultiResult: [Result: "bye!"], 'leaf00.cmh': MultiResult: [Result: "bye!"], 'leaf01.cmh': MultiResult: [Result: "bye!"], 'host1.bma': MultiResult: [Result: "bye!"], 'host2.bma': MultiResult: [Result: "bye!"], 'spine00.bma': MultiResult: [Result: "bye!"], 'spine01.bma': MultiResult: [Result: "bye!"], 'leaf00.bma': MultiResult: [Result: "bye!"], 'leaf01.bma': MultiResult: [Result: "bye!"]}

The first thing you probably noticed is that we got all those messages on screen printed for us. That was done by our processor PrintResult. You probably also noticed we got the AggregatedResult back but we didn’t even bother saving it into a variable as we don’t needed it here.

Now, let’s see if SaveResultToDict did something to the dictionary data:

  1. [6]:
  1. import json
  2. print(json.dumps(data, indent=4))
  1. {
  2.     "hi!": {
  3.         "started": true,
  4.         "host1.cmh": {
  5.             "completed": true,
  6.             "result": "hi! my name is host1.cmh"
  7.         },
  8.         "host2.cmh": {
  9.             "completed": true,
  10.             "result": "hi! my name is host2.cmh"
  11.         },
  12.         "spine00.cmh": {
  13.             "completed": true,
  14.             "result": "hi! my name is spine00.cmh"
  15.         },
  16.         "spine01.cmh": {
  17.             "completed": true,
  18.             "result": "hi! my name is spine01.cmh"
  19.         },
  20.         "leaf00.cmh": {
  21.             "completed": true,
  22.             "result": "hi! my name is leaf00.cmh"
  23.         },
  24.         "leaf01.cmh": {
  25.             "completed": true,
  26.             "result": "hi! my name is leaf01.cmh"
  27.         },
  28.         "host1.bma": {
  29.             "completed": true,
  30.             "result": "hi! my name is host1.bma"
  31.         },
  32.         "host2.bma": {
  33.             "completed": true,
  34.             "result": "hi! my name is host2.bma"
  35.         },
  36.         "spine00.bma": {
  37.             "completed": true,
  38.             "result": "hi! my name is spine00.bma"
  39.         },
  40.         "spine01.bma": {
  41.             "completed": true,
  42.             "result": "hi! my name is spine01.bma"
  43.         },
  44.         "leaf00.bma": {
  45.             "completed": true,
  46.             "result": "hi! my name is leaf00.bma"
  47.         },
  48.         "leaf01.bma": {
  49.             "completed": true,
  50.             "result": "hi! my name is leaf01.bma"
  51.         },
  52.         "completed": true
  53.     },
  54.     "bye!": {
  55.         "started": true,
  56.         "host1.cmh": {
  57.             "completed": true,
  58.             "result": "bye! my name is host1.cmh"
  59.         },
  60.         "host2.cmh": {
  61.             "completed": true,
  62.             "result": "bye! my name is host2.cmh"
  63.         },
  64.         "spine00.cmh": {
  65.             "completed": true,
  66.             "result": "bye! my name is spine00.cmh"
  67.         },
  68.         "spine01.cmh": {
  69.             "completed": true,
  70.             "result": "bye! my name is spine01.cmh"
  71.         },
  72.         "leaf00.cmh": {
  73.             "completed": true,
  74.             "result": "bye! my name is leaf00.cmh"
  75.         },
  76.         "leaf01.cmh": {
  77.             "completed": true,
  78.             "result": "bye! my name is leaf01.cmh"
  79.         },
  80.         "host1.bma": {
  81.             "completed": true,
  82.             "result": "bye! my name is host1.bma"
  83.         },
  84.         "host2.bma": {
  85.             "completed": true,
  86.             "result": "bye! my name is host2.bma"
  87.         },
  88.         "spine00.bma": {
  89.             "completed": true,
  90.             "result": "bye! my name is spine00.bma"
  91.         },
  92.         "spine01.bma": {
  93.             "completed": true,
  94.             "result": "bye! my name is spine01.bma"
  95.         },
  96.         "leaf00.bma": {
  97.             "completed": true,
  98.             "result": "bye! my name is leaf00.bma"
  99.         },
  100.         "leaf01.bma": {
  101.             "completed": true,
  102.             "result": "bye! my name is leaf01.bma"
  103.         },
  104.         "completed": true
  105.     }
  106. }

As you can see, performing various actions on the results becomes quite easy thanks to the processors. You still get the result back but thanks to this plugins you may not needed them anymore.

Ideas

What other things could be done with processors?

  1. Send events to slack/IRC/logging_system

  2. Keep the user informed of what’s going on without having them to wait for the completion of all the hosts (particularly interesting if you have lots of devices)

  3. Page someone/raise an alert if a given task fails

  4. etc…