Advanced filtering

In this tutorial we are going to see how to use the F object to do advanced filtering of hosts. Let’s start by initiating nornir and looking at the inventory:

  1. [1]:
  1. from nornir import InitNornir
  2. from nornir.core.filter import F
  4. nr = InitNornir(config_file="advanced_filtering/config.yaml")
  1. [2]:
  1. %cat advanced_filtering/inventory/hosts.yaml
  1. ---
  2. cat:
  3.     groups:
  4.         - terrestrial
  5.         - mammal
  6.     data:
  7.         domestic: true
  8.         diet: omnivore
  9.         additional_data:
  10.             lifespan: 17
  11.             famous_members:
  12.                 - garfield
  13.                 - felix
  14.                 - grumpy
  16. bat:
  17.     groups:
  18.         - terrestrial
  19.         - mammal
  20.     data:
  21.         domestic: false
  22.         fly: true
  23.         diet: carnivore
  24.         additional_data:
  25.             lifespan: 15
  26.             famous_members:
  27.                 - batman
  28.                 - count chocula
  29.                 - nosferatu
  31. eagle:
  32.     groups:
  33.         - terrestrial
  34.         - bird
  35.     data:
  36.         domestic: false
  37.         diet: carnivore
  38.         additional_data:
  39.             lifespan: 50
  40.             famous_members:
  41.                 - thorondor
  42.                 - sam
  44. canary:
  45.     groups:
  46.         - terrestrial
  47.         - bird
  48.     data:
  49.         domestic: true
  50.         diet: herbivore
  51.         additional_data:
  52.             lifespan: 15
  53.             famous_members:
  54.                 - tweetie
  56. caterpillaer:
  57.     groups:
  58.         - terrestrial
  59.         - invertebrate
  60.     data:
  61.         domestic: false
  62.         diet: herbivore
  63.         additional_data:
  64.             lifespan: 1
  65.             famous_members:
  66.                 - Hookah-Smoking
  68. octopus:
  69.     groups:
  70.         - marine
  71.         - invertebrate
  72.     data:
  73.         domestic: false
  74.         diet: carnivore
  75.         additional_data:
  76.             lifespan: 1
  77.             famous_members:
  78.                 - sharktopus
  1. [3]:
  1. %cat advanced_filtering/inventory/groups.yaml
  1. ---
  2. mammal:
  3.     data:
  4.         reproduction: birth
  5.         fly: false
  7. bird:
  8.     data:
  9.         reproduction: eggs
  10.         fly: true
  12. invertebrate:
  13.     data:
  14.         reproduction: mitosis
  15.         fly: false
  17. terrestrial: {}
  18. marine: {}

As you can see we have built ourselves a collection of animals with different properties. The F object let’s you access the magic methods of each types by just prepeding two underscores and the the name of the magic method. For instance, if you want to check if a list contains a particular element you can just prepend __contains. Let’s use this feature to retrieve all the animals that belong to the group bird:

  1. [4]:
  1. birds = nr.filter(F(groups__contains="bird"))
  2. print(birds.inventory.hosts.keys())
  1. dict_keys(['eagle', 'canary'])

We can also invert the F object by prepending ~:

  1. [5]:
  1. not_birds = nr.filter(~F(groups__contains="bird"))
  2. print(not_birds.inventory.hosts.keys())
  1. dict_keys(['cat', 'bat', 'caterpillaer', 'octopus'])

We can also combine F objects and perform AND and OR operations with the symbols & and | (pipe) respectively:

  1. [6]:
  1. domestic_or_bird = nr.filter(F(groups__contains="bird") | F(domestic=True))
  2. print(domestic_or_bird.inventory.hosts.keys())
  1. dict_keys(['cat', 'eagle', 'canary'])
  1. [7]:
  1. domestic_mammals = nr.filter(F(groups__contains="mammal") & F(domestic=True))
  2. print(domestic_mammals.inventory.hosts.keys())
  1. dict_keys(['cat'])

As expected, you can combine all of the symbols:

  1. [8]:
  1. flying_not_carnivore = nr.filter(F(fly=True) & ~F(diet="carnivore"))
  2. print(flying_not_carnivore.inventory.hosts.keys())
  1. dict_keys(['canary'])

You can also access nested data the same way you access magic methods, by appending two underscores and the data you want to access. You can keep building on this as much as needed and even access the magic methods of the nested data. For instance, let’s get the animals that have a lifespan greater or equal than 15:

  1. [9]:
  1. long_lived = nr.filter(F(additional_data__lifespan__ge=15))
  2. print(long_lived.inventory.hosts.keys())
  1. dict_keys(['cat', 'bat', 'eagle', 'canary'])

There are two extra facilities to help you working with lists; any and all. Those facilities let’s you send a list of elements and get the objects that has either any of the members or all of them. For instance:

  1. [10]:
  1. marine_and_invertebrates = nr.filter(F(groups__all=["marine", "invertebrate"]))
  2. print(marine_and_invertebrates.inventory.hosts.keys())
  1. dict_keys(['octopus'])
  1. [11]:
  1. bird_or_invertebrates = nr.filter(F(groups__any=["bird", "invertebrate"]))
  2. print(bird_or_invertebrates.inventory.hosts.keys())
  1. dict_keys(['eagle', 'canary', 'caterpillaer', 'octopus'])