First class iterators

There are 2 kinds of iterators in Nim: inline and closure iterators. An inline iterator is an iterator that’s always inlined by the compiler leading to zero overhead for the abstraction, but may result in a heavy increase in code size.

Caution: the body of a for loop over an inline iterator is inlined into each yield statement appearing in the iterator code, so ideally the code should be refactored to contain a single yield when possible to avoid code bloat.

Inline iterators are second class citizens; They can be passed as parameters only to other inlining code facilities like templates, macros and other inline iterators.

In contrast to that, a closure iterator can be passed around more freely:

  1. iterator count0(): int {.closure.} =
  2.   yield 0
  4. iterator count2(): int {.closure.} =
  5.   var x = 1
  6.   yield x
  7.   inc x
  8.   yield x
  10. proc invoke(iter: iterator(): int {.closure.}) =
  11.   for x in iter(): echo x
  13. invoke(count0)
  14. invoke(count2)

Closure iterators and inline iterators have some restrictions:

  1. For now, a closure iterator cannot be executed at compile time.
  2. return is allowed in a closure iterator but not in an inline iterator (but rarely useful) and ends the iteration.
  3. Neither inline nor closure iterators can be recursive.
  4. Neither inline nor closure iterators have the special result variable.
  5. Closure iterators are not supported by the js backend.

Iterators that are neither marked {.closure.} nor {.inline.} explicitly default to being inline, but this may change in future versions of the implementation.

The iterator type is always of the calling convention closure implicitly; the following example shows how to use iterators to implement a collaborative tasking system:

  1. # simple tasking:
  2. type
  3.   Task = iterator (ticker: int)
  5. iterator a1(ticker: int) {.closure.} =
  6.   echo "a1: A"
  7.   yield
  8.   echo "a1: B"
  9.   yield
  10.   echo "a1: C"
  11.   yield
  12.   echo "a1: D"
  14. iterator a2(ticker: int) {.closure.} =
  15.   echo "a2: A"
  16.   yield
  17.   echo "a2: B"
  18.   yield
  19.   echo "a2: C"
  21. proc runTasks(t: varargs[Task]) =
  22.   var ticker = 0
  23.   while true:
  24.     let x = t[ticker mod t.len]
  25.     if finished(x): break
  26.     x(ticker)
  27.     inc ticker
  29. runTasks(a1, a2)

The builtin system.finished can be used to determine if an iterator has finished its operation; no exception is raised on an attempt to invoke an iterator that has already finished its work.

Note that system.finished is error prone to use because it only returns true one iteration after the iterator has finished:

  1. iterator mycount(a, b: int): int {.closure.} =
  2.   var x = a
  3.   while x <= b:
  4.     yield x
  5.     inc x
  7. var c = mycount # instantiate the iterator
  8. while not finished(c):
  9.   echo c(1, 3)
  11. # Produces
  12. 1
  13. 2
  14. 3
  15. 0

Instead this code has to be used:

  1. var c = mycount # instantiate the iterator
  2. while true:
  3.   let value = c(1, 3)
  4.   if finished(c): break # and discard 'value'!
  5.   echo value

It helps to think that the iterator actually returns a pair (value, done) and finished is used to access the hidden done field.

Closure iterators are resumable functions and so one has to provide the arguments to every call. To get around this limitation one can capture parameters of an outer factory proc:

  1. proc mycount(a, b: int): iterator (): int =
  2.   result = iterator (): int =
  3.     var x = a
  4.     while x <= b:
  5.       yield x
  6.       inc x
  8. let foo = mycount(1, 4)
  10. for f in foo():
  11.   echo f