Please support this book: buy it (PDF, EPUB, MOBI) or donate

21. Iterables and iterators

21.1 Overview

ES6 introduces a new mechanism for traversing data: iteration. Two concepts are central to iteration:

  • An iterable is a data structure that wants to make its elements accessible to the public. It does so by implementing a method whose key is Symbol.iterator. That method is a factory for iterators.
  • An iterator is a pointer for traversing the elements of a data structure (think cursors in databases). Expressed as interfaces in TypeScript notation, these roles look like this:
  1. interface Iterable {
  2.     [Symbol.iterator]() : Iterator;
  3. }
  4. interface Iterator {
  5.     next() : IteratorResult;
  6. }
  7. interface IteratorResult {
  8.     value: any;
  9.     done: boolean;
  10. }

21.1.1 Iterable values

The following values are iterable:

  • Arrays
  • Strings
  • Maps
  • Sets
  • DOM data structures (work in progress) Plain objects are not iterable (why is explained in a dedicated section).

21.1.2 Constructs supporting iteration

Language constructs that access data via iteration:

  • Destructuring via an Array pattern: 
  1.   const [a,b] = new Set(['a', 'b', 'c']);
  • for-of loop: 
  1.   for (const x of ['a', 'b', 'c']) {
  2.       console.log(x);
  3.   }
  • Array.from(): 
  1.   const arr = Array.from(new Set(['a', 'b', 'c']));
  • Spread operator (…): 
  1.   const arr = [...new Set(['a', 'b', 'c'])];
  • Constructors of Maps and Sets: 
  1.   const map = new Map([[false, 'no'], [true, 'yes']]);
  2.   const set = new Set(['a', 'b', 'c']);
  • Promise.all(), Promise.race(): 
  1.   Promise.all(iterableOverPromises).then(···);
  2.   Promise.race(iterableOverPromises).then(···);
  • yield*: 
  1.   yield* anIterable;

21.2 Iterability

The idea of iterability is as follows.

  • Data consumers: JavaScript has language constructs that consume data. For example, for-of loops over values and the spread operator (…) inserts values into Arrays or function calls.
  • Data sources: The data consumers could get their values from a variety of sources. For example, you may want to iterate over the elements of an Array, the key-value entries in a Map or the characters of a string. It’s not practical for every consumer to support all sources, especially because it should be possible to create new sources (e.g. via libraries). Therefore, ES6 introduces the interface Iterable. Data consumers use it, data sources implement it:

21. Iterables and iterators - 图1

Given that JavaScript does not have interfaces, Iterable is more of a convention:

  • Source: A value is considered iterable if it has a method whose key is the symbol Symbol.iterator that returns a so-called iterator. The iterator is an object that returns values via its method next(). We say: it iterates over the items (the content) of the iterable, one per method call.
  • Consumption: Data consumers use the iterator to retrieve the values they are consuming. Let’s see what consumption looks like for an Array arr. First, you create an iterator via the method whose key is Symbol.iterator:
  1. > const arr = ['a', 'b', 'c'];
  2. > const iter = arr[Symbol.iterator]();

Then you call the iterator’s method next() repeatedly to retrieve the items “inside” the Array:

  1. > iter.next()
  2. { value: 'a', done: false }
  3. > iter.next()
  4. { value: 'b', done: false }
  5. > iter.next()
  6. { value: 'c', done: false }
  7. > iter.next()
  8. { value: undefined, done: true }

As you can see, next() returns each item wrapped in an object, as the value of the property value. The boolean property done indicates when the end of the sequence of items has been reached.

Iterable and iterators are part of a so-called protocol (interfaces plus rules for using them) for iteration. A key characteristic of this protocol is that it is sequential: the iterator returns values one at a time. That means that if an iterable data structure is non-linear (such as a tree), iteration will linearize it.

21.3 Iterable data sources

I’ll use the for-of loop (see Chap. “The for-of loop”) to iterate over various kinds of iterable data.

21.3.1 Arrays

Arrays (and Typed Arrays) are iterables over their elements:

  1. for (const x of ['a', 'b']) {
  2.     console.log(x);
  3. }
  4. // Output:
  5. // 'a'
  6. // 'b'

21.3.2 Strings

Strings are iterable, but they iterate over Unicode code points, each of which may comprise one or two JavaScript characters:

  1. for (const x of 'a\uD83D\uDC0A') {
  2.     console.log(x);
  3. }
  4. // Output:
  5. // 'a'
  6. // '\uD83D\uDC0A' (crocodile emoji)

21.3.3 Maps

Maps are iterables over their entries. Each entry is encoded as a [key, value] pair, an Array with two elements. The entries are always iterated over deterministically, in the same order in which they were added to the map.

  1. const map = new Map().set('a', 1).set('b', 2);
  2. for (const pair of map) {
  3.     console.log(pair);
  4. }
  5. // Output:
  6. // ['a', 1]
  7. // ['b', 2]

Note that WeakMaps are not iterable.

21.3.4 Sets

Sets are iterables over their elements (which are iterated over in the same order in which they were added to the Set).

  1. const set = new Set().add('a').add('b');
  2. for (const x of set) {
  3.     console.log(x);
  4. }
  5. // Output:
  6. // 'a'
  7. // 'b'

Note that WeakSets are not iterable.

21.3.5 arguments

Even though the special variable arguments is more or less obsolete in ECMAScript 6 (due to rest parameters), it is iterable:

  1. function printArgs() {
  2.     for (const x of arguments) {
  3.         console.log(x);
  4.     }
  5. }
  6. printArgs('a', 'b');
  7.  
  8. // Output:
  9. // 'a'
  10. // 'b'

21.3.6 DOM data structures

Most DOM data structures will eventually be iterable:

  1. for (const node of document.querySelectorAll('div')) {
  2.     ···
  3. }

Note that implementing this functionality is work in progress. But it is relatively easy to do so, because the symbol Symbol.iterator can’t clash with existing property keys.

21.3.7 Iterable computed data

Not all iterable content does have to come from data structures, it could also be computed on the fly. For example, all major ES6 data structures (Arrays, Typed Arrays, Maps, Sets) have three methods that return iterable objects:

  • entries() returns an iterable over entries encoded as [key, value] Arrays. For Arrays, the values are the Array elements and the keys are their indices. For Sets, each key and value are the same – the Set element.
  • keys() returns an iterable over the keys of the entries.
  • values() returns an iterable over the values of the entries. Let’s see what that looks like. entries() gives you a nice way to get both Array elements and their indices:
  1. const arr = ['a', 'b', 'c'];
  2. for (const pair of arr.entries()) {
  3.     console.log(pair);
  4. }
  5. // Output:
  6. // [0, 'a']
  7. // [1, 'b']
  8. // [2, 'c']

21.3.8 Plain objects are not iterable

Plain objects (as created by object literals) are not iterable:

  1. for (const x of {}) { // TypeError
  2.     console.log(x);
  3. }

Why aren’t objects iterable over properties, by default? The reasoning is as follows. There are two levels at which you can iterate in JavaScript:

  • The program level: iterating over properties means examining the structure of the program.

  • The data level: iterating over a data structure means examining the data managed by the program. Making iteration over properties the default would mean mixing those levels, which would have two disadvantages:

  • You can’t iterate over the properties of data structures.

  • Once you iterate over the properties of an object, turning that object into a data structure would break your code. If engines were to implement iterability via a method Object.prototypeSymbol.iterator then there would be an additional caveat: Objects created via Object.create(null) wouldn’t be iterable, because Object.prototype is not in their prototype chain.

It is important to remember that iterating over the properties of an object is mainly interesting if you use objects as Maps1. But we only do that in ES5 because we have no better alternative. In ECMAScript 6, we have the built-in data structure Map.

21.3.8.1 How to iterate over properties

The proper (and safe) way to iterate over properties is via a tool function. For example, via objectEntries(), whose implementation is shown later (future ECMAScript versions may have something similar built in):

  1. const obj = { first: 'Jane', last: 'Doe' };
  2.  
  3. for (const [key,value] of objectEntries(obj)) {
  4.     console.log(`${key}: ${value}`);
  5. }
  6.  
  7. // Output:
  8. // first: Jane
  9. // last: Doe

21.4 Iterating language constructs

The following ES6 language constructs make use of the iteration protocol:

  • Destructuring via an Array pattern
  • for-of loop
  • Array.from()
  • Spread operator (…)
  • Constructors of Maps and Sets
  • Promise.all(), Promise.race()
  • yield* The next sections describe each one of them in detail.

21.4.1 Destructuring via an Array pattern

Destructuring via Array patterns works for any iterable:

  1. const set = new Set().add('a').add('b').add('c');
  2.  
  3. const [x,y] = set;
  4.     // x='a'; y='b'
  5.  
  6. const [first, ...rest] = set;
  7.     // first='a'; rest=['b','c'];

21.4.2 The for-of loop

for-of is a new loop in ECMAScript 6. It’s basic form looks like this:

  1. for (const x of iterable) {
  2.     ···
  3. }

For more information, consult Chap. “The for-of loop”.

Note that the iterability of iterable is required, otherwise for-of can’t loop over a value. That means that non-iterable values must be converted to something iterable. For example, via Array.from().

21.4.3 Array.from()

Array.from() converts iterable and Array-like values to Arrays. It is also available for typed Arrays.

  1. > Array.from(new Map().set(false, 'no').set(true, 'yes'))
  2. [[false,'no'], [true,'yes']]
  3. > Array.from({ length: 2, 0: 'hello', 1: 'world' })
  4. ['hello', 'world']

For more information on Array.from(), consult the chapter on Arrays.

21.4.4 The spread operator (…)

The spread operator inserts the values of an iterable into an Array:

  1. > const arr = ['b', 'c'];
  2. > ['a', ...arr, 'd']
  3. ['a', 'b', 'c', 'd']

That means that it provides you with a compact way to convert any iterable to an Array:

  1. const arr = [...iterable];

The spread operator also turns an iterable into the arguments of a function, method or constructor call:

  1. > Math.max(...[-1, 8, 3])
  2. 8

21.4.5 Maps and Sets

The constructor of a Map turns an iterable over [key, value] pairs into a Map:

  1. > const map = new Map([['uno', 'one'], ['dos', 'two']]);
  2. > map.get('uno')
  3. 'one'
  4. > map.get('dos')
  5. 'two'

The constructor of a Set turns an iterable over elements into a Set:

  1. > const set = new Set(['red', 'green', 'blue']);
  2. > set.has('red')
  3. true
  4. > set.has('yellow')
  5. false

The constructors of WeakMap and WeakSet work similarly. Furthermore, Maps and Sets are iterable themselves (WeakMaps and WeakSets aren’t), which means that you can use their constructors to clone them.

21.4.6 Promises

Promise.all() and Promise.race() accept iterables over Promises:

  1. Promise.all(iterableOverPromises).then(···);
  2. Promise.race(iterableOverPromises).then(···);

21.4.7 yield*

yield* is an operator that is only available inside generators. It yields all items iterated over by an iterable.

  1. function* yieldAllValuesOf(iterable) {
  2.     yield* iterable;
  3. }

The most important use case for yield* is to recursively call a generator (which produces something iterable).

21.5 Implementing iterables

In this section, I explain in detail how to implement iterables. Note that ES6 generators are usually much more convenient for this task than doing so “manually”.

The iteration protocol looks as follows.

21. Iterables and iterators - 图2

An object becomes iterable (“implements” the interface Iterable) if it has a method (own or inherited) whose key is Symbol.iterator. That method must return an iterator, an object that iterates over the items “inside” the iterable via its method next().

In TypeScript notation, the interfaces for iterables and iterators look as follows2.

  1. interface Iterable {
  2.     [Symbol.iterator]() : Iterator;
  3. }
  4. interface Iterator {
  5.     next() : IteratorResult;
  6.     return?(value? : any) : IteratorResult;
  7. }
  8. interface IteratorResult {
  9.     value: any;
  10.     done: boolean;
  11. }

return() is an optional method that we’ll get to later3. Let’s first implement a dummy iterable to get a feeling for how iteration works.

  1. const iterable = {
  2.     [Symbol.iterator]() {
  3.         let step = 0;
  4.         const iterator = {
  5.             next() {
  6.                 if (step <= 2) {
  7.                     step++;
  8.                 }
  9.                 switch (step) {
  10.                     case 1:
  11.                         return { value: 'hello', done: false };
  12.                     case 2:
  13.                         return { value: 'world', done: false };
  14.                     default:
  15.                         return { value: undefined, done: true };
  16.                 }
  17.             }
  18.         };
  19.         return iterator;
  20.     }
  21. };

Let’s check that iterable is, in fact, iterable:

  1. for (const x of iterable) {
  2.     console.log(x);
  3. }
  4. // Output:
  5. // hello
  6. // world

The code executes three steps, with the counter step ensuring that everything happens in the right order. First, we return the value 'hello', then the value 'world' and then we indicate that the end of the iteration has been reached. Each item is wrapped in an object with the properties:

  • value which holds the actual item and
  • done which is a boolean flag that indicates whether the end has been reached, yet. You can omit done if it is false and value if it is undefined. That is, the switch statement could be written as follows.
  1. switch (step) {
  2.     case 1:
  3.         return { value: 'hello' };
  4.     case 2:
  5.         return { value: 'world' };
  6.     default:
  7.         return { done: true };
  8. }

As is explained in the the chapter on generators, there are cases where you want even the last item with done: true to have a value. Otherwise, next() could be simpler and return items directly (without wrapping them in objects). The end of iteration would then be indicated via a special value (e.g., a symbol).

Let’s look at one more implementation of an iterable. The function iterateOver() returns an iterable over the arguments that are passed to it:

  1. function iterateOver(...args) {
  2.     let index = 0;
  3.     const iterable = {
  4.         [Symbol.iterator]() {
  5.             const iterator = {
  6.                 next() {
  7.                     if (index < args.length) {
  8.                         return { value: args[index++] };
  9.                     } else {
  10.                         return { done: true };
  11.                     }
  12.                 }
  13.             };
  14.             return iterator;
  15.         }
  16.     }
  17.     return iterable;
  18. }
  19.  
  20. // Using `iterateOver()`:
  21. for (const x of iterateOver('fee', 'fi', 'fo', 'fum')) {
  22.     console.log(x);
  23. }
  24.  
  25. // Output:
  26. // fee
  27. // fi
  28. // fo
  29. // fum

21.5.1 Iterators that are iterable

The previous function can be simplified if the iterable and the iterator are the same object:

  1. function iterateOver(...args) {
  2.     let index = 0;
  3.     const iterable = {
  4.         [Symbol.iterator]() {
  5.             return this;
  6.         },
  7.         next() {
  8.             if (index < args.length) {
  9.                 return { value: args[index++] };
  10.             } else {
  11.                 return { done: true };
  12.             }
  13.         },
  14.     };
  15.     return iterable;
  16. }

Even if the original iterable and the iterator are not the same object, it is still occasionally useful if an iterator has the following method (which also makes it an iterable):

  1. [Symbol.iterator]() {
  2.     return this;
  3. }

All built-in ES6 iterators follow this pattern (via a common prototype, see the chapter on generators). For example, the default iterator for Arrays:

  1. > const arr = [];
  2. > const iterator = arr[Symbol.iterator]();
  3. > iterator[Symbol.iterator]() === iterator
  4. true

Why is it useful if an iterator is also an iterable? for-of only works for iterables, not for iterators. Because Array iterators are iterable, you can continue an iteration in another loop:

  1. const arr = ['a', 'b'];
  2. const iterator = arr[Symbol.iterator]();
  3.  
  4. for (const x of iterator) {
  5.     console.log(x); // a
  6.     break;
  7. }
  8.  
  9. // Continue with same iterator:
  10. for (const x of iterator) {
  11.     console.log(x); // b
  12. }

One use case for continuing an iteration is that you can remove initial items (e.g. a header) before processing the actual content via for-of.

21.5.2 Optional iterator methods: return() and throw()

Two iterator methods are optional:

  • return() gives an iterator the opportunity to clean up if an iteration ends prematurely.
  • throw() is about forwarding a method call to a generator that is iterated over via yield*. It is explained in the chapter on generators.
21.5.2.1 Closing iterators via return()

As mentioned before, the optional iterator method return() is about letting an iterator clean up if it wasn’t iterated over until the end. It closes an iterator. In for-of loops, premature (or abrupt, in spec language) termination can be caused by:

  • break
  • continue (if you continue an outer loop, continue acts like a break)
  • throw
  • return In each of these cases, for-of lets the iterator know that the loop won’t finish. Let’s look at an example, a function readLinesSync that returns an iterable of text lines in a file and would like to close that file no matter what happens:
  1. function readLinesSync(fileName) {
  2.     const file = ···;
  3.     return {
  4.         ···
  5.         next() {
  6.             if (file.isAtEndOfFile()) {
  7.                 file.close();
  8.                 return { done: true };
  9.             }
  10.             ···
  11.         },
  12.         return() {
  13.             file.close();
  14.             return { done: true };
  15.         },
  16.     };
  17. }

Due to return(), the file will be properly closed in the following loop:

  1. // Only print first line
  2. for (const line of readLinesSync(fileName)) {
  3.     console.log(x);
  4.     break;
  5. }

The return() method must return an object. That is due to how generators handle the return statement and will be explained in the chapter on generators.

The following constructs close iterators that aren’t completely “drained”:

  • for-of
  • yield*
  • Destructuring
  • Array.from()
  • Map(), Set(), WeakMap(), WeakSet()
  • Promise.all(), Promise.race() A later section has more information on closing iterators.

21.6 More examples of iterables

In this section, we look at a few more examples of iterables. Most of these iterables are easier to implement via generators. The chapter on generators shows how.

21.6.1 Tool functions that return iterables

Tool functions and methods that return iterables are just as important as iterable data structures. The following is a tool function for iterating over the own properties of an object.

  1. function objectEntries(obj) {
  2.     let index = 0;
  3.  
  4.     // In ES6, you can use strings or symbols as property keys,
  5.     // Reflect.ownKeys() retrieves both
  6.     const propKeys = Reflect.ownKeys(obj);
  7.  
  8.     return {
  9.         [Symbol.iterator]() {
  10.             return this;
  11.         },
  12.         next() {
  13.             if (index < propKeys.length) {
  14.                 const key = propKeys[index];
  15.                 index++;
  16.                 return { value: [key, obj[key]] };
  17.             } else {
  18.                 return { done: true };
  19.             }
  20.         }
  21.     };
  22. }
  23.  
  24. const obj = { first: 'Jane', last: 'Doe' };
  25. for (const [key,value] of objectEntries(obj)) {
  26.     console.log(`${key}: ${value}`);
  27. }
  28.  
  29. // Output:
  30. // first: Jane
  31. // last: Doe

Another option is to use an iterator instead of an index to traverse the Array with the property keys:

  1. function objectEntries(obj) {
  2.     let iter = Reflect.ownKeys(obj)[Symbol.iterator]();
  3.  
  4.     return {
  5.         [Symbol.iterator]() {
  6.             return this;
  7.         },
  8.         next() {
  9.             let { done, value: key } = iter.next();
  10.             if (done) {
  11.                 return { done: true };
  12.             }
  13.             return { value: [key, obj[key]] };
  14.         }
  15.     };
  16. }

21.6.2 Combinators for iterables

Combinators4 are functions that combine existing iterables to create new ones.

21.6.2.1 take(n, iterable)

Let’s start with the combinator function take(n, iterable), which returns an iterable over the first n items of iterable.

  1. function take(n, iterable) {
  2.     const iter = iterable[Symbol.iterator]();
  3.     return {
  4.         [Symbol.iterator]() {
  5.             return this;
  6.         },
  7.         next() {
  8.             if (n > 0) {
  9.                 n--;
  10.                 return iter.next();
  11.             } else {
  12.                 return { done: true };
  13.             }
  14.         }
  15.     };
  16. }
  17. const arr = ['a', 'b', 'c', 'd'];
  18. for (const x of take(2, arr)) {
  19.     console.log(x);
  20. }
  21. // Output:
  22. // a
  23. // b
21.6.2.2 zip(…iterables)

zip turns n iterables into an iterable of n-tuples (encoded as Arrays of length n).

  1. function zip(...iterables) {
  2.     const iterators = iterables.map(i => i[Symbol.iterator]());
  3.     let done = false;
  4.     return {
  5.         [Symbol.iterator]() {
  6.             return this;
  7.         },
  8.         next() {
  9.             if (!done) {
  10.                 const items = iterators.map(i => i.next());
  11.                 done = items.some(item => item.done);
  12.                 if (!done) {
  13.                     return { value: items.map(i => i.value) };
  14.                 }
  15.                 // Done for the first time: close all iterators
  16.                 for (const iterator of iterators) {
  17.                     if (typeof iterator.return === 'function') {
  18.                         iterator.return();
  19.                     }
  20.                 }
  21.             }
  22.             // We are done
  23.             return { done: true };
  24.         }
  25.     }
  26. }

As you can see, the shortest iterable determines the length of the result:

  1. const zipped = zip(['a', 'b', 'c'], ['d', 'e', 'f', 'g']);
  2. for (const x of zipped) {
  3.     console.log(x);
  4. }
  5. // Output:
  6. // ['a', 'd']
  7. // ['b', 'e']
  8. // ['c', 'f']

21.6.3 Infinite iterables

Some iterable may never be done.

  1. function naturalNumbers() {
  2.     let n = 0;
  3.     return {
  4.         [Symbol.iterator]() {
  5.             return this;
  6.         },
  7.         next() {
  8.             return { value: n++ };
  9.         }
  10.     }
  11. }

With an infinite iterable, you must not iterate over “all” of it. For example, by breaking from a for-of loop:

  1. for (const x of naturalNumbers()) {
  2.     if (x > 2) break;
  3.     console.log(x);
  4. }

Or by only accessing the beginning of an infinite iterable:

  1. const [a, b, c] = naturalNumbers();
  2.     // a=0; b=1; c=2;

Or by using a combinator. take() is one possibility:

  1. for (const x of take(3, naturalNumbers())) {
  2.     console.log(x);
  3. }
  4. // Output:
  5. // 0
  6. // 1
  7. // 2

The “length” of the iterable returned by zip() is determined by its shortest input iterable. That means that zip() and naturalNumbers() provide you with the means to number iterables of arbitrary (finite) length:

  1. const zipped = zip(['a', 'b', 'c'], naturalNumbers());
  2. for (const x of zipped) {
  3.     console.log(x);
  4. }
  5. // Output:
  6. // ['a', 0]
  7. // ['b', 1]
  8. // ['c', 2]

21.7 FAQ: iterables and iterators

21.7.1 Isn’t the iteration protocol slow?

You may be worried about the iteration protocol being slow, because a new object is created for each invocation of next(). However, memory management for small objects is fast in modern engines and in the long run, engines can optimize iteration so that no intermediate objects need to be allocated. A thread on es-discuss has more information.

21.7.2 Can I reuse the same object several times?

In principle, nothing prevents an iterator from reusing the same iteration result object several times – I’d expect most things to work well. However, there will be problems if a client caches iteration results:

  1. const iterationResults = [];
  2. const iterator = iterable[Symbol.iterator]();
  3. let iterationResult;
  4. while (!(iterationResult = iterator.next()).done) {
  5.     iterationResults.push(iterationResult);
  6. }

If an iterator reuses its iteration result object, iterationResults will, in general, contain the same object multiple times.

21.7.3 Why doesn’t ECMAScript 6 have iterable combinators?

You may be wondering why ECMAScript 6 does not have iterable combinators, tools for working with iterables or for creating iterables. That is because the plans are to proceed in two steps:

  • Step 1: standardize an iteration protocol.
  • Step 2: wait for libraries based on that protocol. Eventually, one such library or pieces from several libraries will be added to the JavaScript standard library.

If you want to get an impression of what such a library could look like, take a look at the standard Python module itertools.

21.7.4 Aren’t iterables difficult to implement?

Yes, iterables are difficult to implement – if you implement them manually. The next chapter will introduce generators that help with this task (among other things).

21.8 The ECMAScript 6 iteration protocol in depth

The iteration protocol comprises the following interfaces (I have omitted throw() from Iterator, which is only supported by yield* and optional there):

  1. interface Iterable {
  2.     [Symbol.iterator]() : Iterator;
  3. }
  4. interface Iterator {
  5.     next() : IteratorResult;
  6.     return?(value? : any) : IteratorResult;
  7. }
  8. interface IteratorResult {
  9.     value : any;
  10.     done : boolean;
  11. }

21.8.1 Iteration

Rules for next():

  • As long as the iterator still has values x to produce, next() returns objects { value: x, done: false }.
  • After the last value was iterated over, next() should always return an object whose property done is true.
21.8.1.1 The IteratorResult

The property done of an iterator result doesn’t have to be true or false, truthy or falsy is enough. All built-in language mechanisms let you omit done: false.

21.8.1.2 Iterables that return fresh iterators versus those that always return the same iterator

Some iterables produce a new iterator each time they are asked for one. For example, Arrays:

  1. function getIterator(iterable) {
  2.     return iterable[Symbol.iterator]();
  3. }
  4.  
  5. const iterable = ['a', 'b'];
  6. console.log(getIterator(iterable) === getIterator(iterable)); // false

Other iterables return the same iterator each time. For example, generator objects:

  1. function* elements() {
  2.     yield 'a';
  3.     yield 'b';
  4. }
  5. const iterable = elements();
  6. console.log(getIterator(iterable) === getIterator(iterable)); // true

Whether an iterable produces a fresh iterators or not matter when you iterate over the same iterable multiple times. For example, via the following function:

  1. function iterateTwice(iterable) {
  2.     for (const x of iterable) {
  3.         console.log(x);
  4.     }
  5.     for (const x of iterable) {
  6.         console.log(x);
  7.     }
  8. }

With fresh iterators, you can iterate over the same iterable multiple times:

  1. iterateTwice(['a', 'b']);
  2. // Output:
  3. // a
  4. // b
  5. // a
  6. // b

If the same iterator is returned each time, you can’t:

  1. iterateTwice(elements());
  2. // Output:
  3. // a
  4. // b

Note that each iterator in the standard library is also an iterable. Its method Symbol.iterator return this, meaning that it always returns the same iterator (itself).

21.8.2 Closing iterators

The iteration protocol distinguishes two ways of finishing an iterator:

  • Exhaustion: the regular way of finishing an iterator is by retrieving all of its values. That is, one calls next() until it returns an object whose property done is true.

  • Closing: by calling return(), you tell the iterator that you don’t intend to call next(), anymore. Rules for calling return():

  • return() is an optional method, not all iterators have it. Iterators that do have it are called closable.

  • return() should only be called if an iterator hasn’t be exhausted. For example, for-of calls return() whenever it is left “abruptly” (before it is finished). The following operations cause abrupt exits: break, continue (with a label of an outer block), return, throw. Rules for implementing return():

  • The method call return(x) should normally produce the object { done: true, value: x }, but language mechanisms only throw an error (source in spec) if the result isn’t an object.

  • After return() was called, the objects returned by next() should be done, too. The following code illustrates that the for-of loop calls return() if it is aborted before it receives a done iterator result. That is, return() is even called if you abort after receiving the last value. This is subtle and you have to be careful to get it right when you iterate manually or implement iterators.
  1. function createIterable() {
  2.     let done = false;
  3.     const iterable = {
  4.         [Symbol.iterator]() {
  5.             return this;
  6.         },
  7.         next() {
  8.             if (!done) {
  9.                 done = true;
  10.                 return { done: false, value: 'a' };
  11.             } else {
  12.                 return { done: true, value: undefined };
  13.             }
  14.         },
  15.         return() {
  16.             console.log('return() was called!');
  17.         },
  18.     };
  19.     return iterable;
  20. }
  21. for (const x of createIterable()) {
  22.     console.log(x);
  23.     // There is only one value in the iterable and
  24.     // we abort the loop after receiving it
  25.     break;
  26. }
  27. // Output:
  28. // a
  29. // return() was called!
21.8.2.1 Closable iterators

An iterator is closable if it has a method return(). Not all iterators are closable. For example, Array iterators are not:

  1. > let iterable = ['a', 'b', 'c'];
  2. > const iterator = iterable[Symbol.iterator]();
  3. > 'return' in iterator
  4. false

Generator objects are closable by default. For example, the ones returned by the following generator function:

  1. function* elements() {
  2.     yield 'a';
  3.     yield 'b';
  4.     yield 'c';
  5. }

If you invoke return() on the result of elements(), iteration is finished:

  1. > const iterator = elements();
  2. > iterator.next()
  3. { value: 'a', done: false }
  4. > iterator.return()
  5. { value: undefined, done: true }
  6. > iterator.next()
  7. { value: undefined, done: true }

If an iterator is not closable, you can continue iterating over it after an abrupt exit (such as the one in line A) from a for-of loop:

  1. function twoLoops(iterator) {
  2.     for (const x of iterator) {
  3.         console.log(x);
  4.         break; // (A)
  5.     }
  6.     for (const x of iterator) {
  7.         console.log(x);
  8.     }
  9. }
  10. function getIterator(iterable) {
  11.     return iterable[Symbol.iterator]();
  12. }
  13.  
  14. twoLoops(getIterator(['a', 'b', 'c']));
  15. // Output:
  16. // a
  17. // b
  18. // c

Conversely, elements() returns a closable iterator and the second loop inside twoLoops() doesn’t have anything to iterate over:

  1. twoLoops(elements());
  2. // Output:
  3. // a
21.8.2.2 Preventing iterators from being closed

The following class is a generic solution for preventing iterators from being closed. It does so by wrapping the iterator and forwarding all method calls except return().

  1. class PreventReturn {
  2.     constructor(iterator) {
  3.         this.iterator = iterator;
  4.     }
  5.     /** Must also be iterable, so that for-of works */
  6.     [Symbol.iterator]() {
  7.         return this;
  8.     }
  9.     next() {
  10.         return this.iterator.next();
  11.     }
  12.     return(value = undefined) {
  13.         return { done: false, value };
  14.     }
  15.     // Not relevant for iterators: `throw()`
  16. }

If we use PreventReturn, the result of the generator elements() won’t be closed after the abrupt exit in the first loop of twoLoops().

  1. function* elements() {
  2.     yield 'a';
  3.     yield 'b';
  4.     yield 'c';
  5. }
  6. function twoLoops(iterator) {
  7.     for (const x of iterator) {
  8.         console.log(x);
  9.         break; // abrupt exit
  10.     }
  11.     for (const x of iterator) {
  12.         console.log(x);
  13.     }
  14. }
  15. twoLoops(elements());
  16. // Output:
  17. // a
  18.  
  19. twoLoops(new PreventReturn(elements()));
  20. // Output:
  21. // a
  22. // b
  23. // c

There is another way of making generators unclosable: All generator objects produced by the generator function elements() have the prototype object elements.prototype. Via elements.prototype, you can hide the default implementation of return() (which resides in a prototype of elements.prototype) as follows:

  1. // Make generator object unclosable
  2. // Warning: may not work in transpilers
  3. elements.prototype.return = undefined;
  4.  
  5. twoLoops(elements());
  6. // Output:
  7. // a
  8. // b
  9. // c
21.8.2.3 Handling clean-up in generators via try-finally

Some generators need to clean up (release allocated resources, close open files, etc.) after iteration over them is finished. Naively, this is how we’d implement it:

  1. function* genFunc() {
  2.     yield 'a';
  3.     yield 'b';
  4.  
  5.     console.log('Performing cleanup');
  6. }

In a normal for-of loop, everything is fine:

  1. for (const x of genFunc()) {
  2.     console.log(x);
  3. }
  4. // Output:
  5. // a
  6. // b
  7. // Performing cleanup

However, if you exit the loop after the first yield, execution seemingly pauses there forever and never reaches the cleanup step:

  1. for (const x of genFunc()) {
  2.     console.log(x);
  3.     break;
  4. }
  5. // Output:
  6. // a

What actually happens is that, whenever one leaves a for-of loop early, for-of sends a return() to the current iterator. That means that the cleanup step isn’t reached because the generator function returns beforehand.

Thankfully, this is easily fixed, by performing the cleanup in a finally clause:

  1. function* genFunc() {
  2.     try {
  3.         yield 'a';
  4.         yield 'b';
  5.     } finally {
  6.         console.log('Performing cleanup');
  7.     }
  8. }

Now everything works as desired:

  1. for (const x of genFunc()) {
  2.     console.log(x);
  3.     break;
  4. }
  5. // Output:
  6. // a
  7. // Performing cleanup

The general pattern for using resources that need to be closed or cleaned up in some manner is therefore:

  1. function* funcThatUsesResource() {
  2.     const resource = allocateResource();
  3.     try {
  4.         ···
  5.     } finally {
  6.         resource.deallocate();
  7.     }
  8. }
21.8.2.4 Handling clean-up in manually implemented iterators
  1. const iterable = {
  2.     [Symbol.iterator]() {
  3.         function hasNextValue() { ··· }
  4.         function getNextValue() { ··· }
  5.         function cleanUp() { ··· }
  6.         let returnedDoneResult = false;
  7.         return {
  8.             next() {
  9.                 if (hasNextValue()) {
  10.                     const value = getNextValue();
  11.                     return { done: false, value: value };
  12.                 } else {
  13.                     if (!returnedDoneResult) {
  14.                         // Client receives first `done` iterator result
  15.                         // => won’t call `return()`
  16.                         cleanUp();
  17.                         returnedDoneResult = true;
  18.                     }
  19.                     return { done: true, value: undefined };
  20.                 }
  21.             },
  22.             return() {
  23.                 cleanUp();
  24.             }
  25.         };
  26.     }
  27. }

Note that you must call cleanUp() when you are going to return a done iterator result for the first time. You must not do it earlier, because then return() may still be called. This can be tricky to get right.

21.8.2.5 Closing iterators you use

If you use iterators, you should close them properly. In generators, you can let for-of do all the work for you:

  1. /**
  2.  * Converts a (potentially infinite) sequence of
  3.  * iterated values into a sequence of length `n`
  4.  */
  5. function* take(n, iterable) {
  6.     for (const x of iterable) {
  7.         if (n <= 0) {
  8.             break; // closes iterable
  9.         }
  10.         n--;
  11.         yield x;
  12.     }
  13. }

If you manage things manually, more work is required:

  1. function* take(n, iterable) {
  2.     const iterator = iterable[Symbol.iterator]();
  3.     while (true) {
  4.         const {value, done} = iterator.next();
  5.         if (done) break; // exhausted
  6.         if (n <= 0) {
  7.             // Abrupt exit
  8.             maybeCloseIterator(iterator);
  9.             break;
  10.         }
  11.         yield value;
  12.         n--;
  13.     }
  14. }
  15. function maybeCloseIterator(iterator) {
  16.     if (typeof iterator.return === 'function') {
  17.         iterator.return();
  18.     }
  19. }

Even more work is necessary if you don’t use generators:

  1. function take(n, iterable) {
  2.     const iter = iterable[Symbol.iterator]();
  3.     return {
  4.         [Symbol.iterator]() {
  5.             return this;
  6.         },
  7.         next() {
  8.             if (n > 0) {
  9.                 n--;
  10.                 return iter.next();
  11.             } else {
  12.                 maybeCloseIterator(iter);
  13.                 return { done: true };
  14.             }
  15.         },
  16.         return() {
  17.             n = 0;
  18.             maybeCloseIterator(iter);
  19.         }
  20.     };
  21. }

21.8.3 Checklist

  • Documenting an iterable: provide the following information.
    • Does it return fresh iterators or the same iterator each time?
    • Are its iterators closable?
  • Implementing an iterator:
    • Clean-up activity must happen if either an iterator is exhausted or if return() is called.
      • In generators, try-finally lets you handle both in a single location.
    • After an iterator was closed via return(), it should not produce any more iterator results via next().
  • Using an iterator manually (versus via for-of etc.):
    • Don’t forget to close the iterator via return, if – and only if – you don’t exhaust it. Getting this right can be tricky.
  • Continuing to iterate over an iterator after an abrupt exit: The iterator must either be unclosable or made unclosable (e.g. via a tool class).