Generator Delegation

In the previous section, we showed calling regular functions from inside a generator, and how that remains a useful technique for abstracting away implementation details (like async Promise flow). But the main drawback of using a normal function for this task is that it has to behave by the normal function rules, which means it cannot pause itself with yield like a generator can.

It may then occur to you that you might try to call one generator from another generator, using our run(..) helper, such as:

  1. function *foo() {
  2.     var r2 = yield request( "http://some.url.2" );
  3.     var r3 = yield request( "http://some.url.3/?v=" + r2 );
  5.     return r3;
  6. }
  8. function *bar() {
  9.     var r1 = yield request( "http://some.url.1" );
  11.     // "delegating" to `*foo()` via `run(..)`
  12.     var r3 = yield run( foo );
  14.     console.log( r3 );
  15. }
  17. run( bar );

We run *foo() inside of *bar() by using our run(..) utility again. We take advantage here of the fact that the run(..) we defined earlier returns a promise which is resolved when its generator is run to completion (or errors out), so if we yield out to a run(..) instance the promise from another run(..) call, it automatically pauses *bar() until *foo() finishes.

But there’s an even better way to integrate calling *foo() into *bar(), and it’s called yield-delegation. The special syntax for yield-delegation is: yield * __ (notice the extra *). Before we see it work in our previous example, let’s look at a simpler scenario:

  1. function *foo() {
  2.     console.log( "`*foo()` starting" );
  3.     yield 3;
  4.     yield 4;
  5.     console.log( "`*foo()` finished" );
  6. }
  8. function *bar() {
  9.     yield 1;
  10.     yield 2;
  11.     yield *foo();    // `yield`-delegation!
  12.     yield 5;
  13. }
  15. var it = bar();
  17. it.next().value;    // 1
  18. it.next().value;    // 2
  19. it.next().value;    // `*foo()` starting
  20.                     // 3
  21. it.next().value;    // 4
  22. it.next().value;    // `*foo()` finished
  23.                     // 5

Note: Similar to a note earlier in the chapter where I explained why I prefer function *foo() .. instead of function* foo() .., I also prefer — differing from most other documentation on the topic — to say yield *foo() instead of yield* foo(). The placement of the * is purely stylistic and up to your best judgment. But I find the consistency of styling attractive.

How does the yield *foo() delegation work?

First, calling foo() creates an iterator exactly as we’ve already seen. Then, yield * delegates/transfers the iterator instance control (of the present *bar() generator) over to this other *foo() iterator.

So, the first two it.next() calls are controlling *bar(), but when we make the third it.next() call, now *foo() starts up, and now we’re controlling *foo() instead of *bar(). That’s why it’s called delegation — *bar() delegated its iteration control to *foo().

As soon as the it iterator control exhausts the entire *foo() iterator, it automatically returns to controlling *bar().

So now back to the previous example with the three sequential Ajax requests:

  1. function *foo() {
  2.     var r2 = yield request( "http://some.url.2" );
  3.     var r3 = yield request( "http://some.url.3/?v=" + r2 );
  5.     return r3;
  6. }
  8. function *bar() {
  9.     var r1 = yield request( "http://some.url.1" );
  11.     // "delegating" to `*foo()` via `yield*`
  12.     var r3 = yield *foo();
  14.     console.log( r3 );
  15. }
  17. run( bar );

The only difference between this snippet and the version used earlier is the use of yield *foo() instead of the previous yield run(foo).

Note: yield * yields iteration control, not generator control; when you invoke the *foo() generator, you’re now yield-delegating to its iterator. But you can actually yield-delegate to any iterable; yield *[1,2,3] would consume the default iterator for the [1,2,3] array value.

Why Delegation?

The purpose of yield-delegation is mostly code organization, and in that way is symmetrical with normal function calling.

Imagine two modules that respectively provide methods foo() and bar(), where bar() calls foo(). The reason the two are separate is generally because the proper organization of code for the program calls for them to be in separate functions. For example, there may be cases where foo() is called standalone, and other places where bar() calls foo().

For all these exact same reasons, keeping generators separate aids in program readability, maintenance, and debuggability. In that respect, yield * is a syntactic shortcut for manually iterating over the steps of *foo() while inside of *bar().

Such manual approach would be especially complex if the steps in *foo() were asynchronous, which is why you’d probably need to use that run(..) utility to do it. And as we’ve shown, yield *foo() eliminates the need for a sub-instance of the run(..) utility (like run(foo)).

Delegating Messages

You may wonder how this yield-delegation works not just with iterator control but with the two-way message passing. Carefully follow the flow of messages in and out, through the yield-delegation:

  1. function *foo() {
  2.     console.log( "inside `*foo()`:", yield "B" );
  4.     console.log( "inside `*foo()`:", yield "C" );
  6.     return "D";
  7. }
  9. function *bar() {
  10.     console.log( "inside `*bar()`:", yield "A" );
  12.     // `yield`-delegation!
  13.     console.log( "inside `*bar()`:", yield *foo() );
  15.     console.log( "inside `*bar()`:", yield "E" );
  17.     return "F";
  18. }
  20. var it = bar();
  22. console.log( "outside:", it.next().value );
  23. // outside: A
  25. console.log( "outside:", it.next( 1 ).value );
  26. // inside `*bar()`: 1
  27. // outside: B
  29. console.log( "outside:", it.next( 2 ).value );
  30. // inside `*foo()`: 2
  31. // outside: C
  33. console.log( "outside:", it.next( 3 ).value );
  34. // inside `*foo()`: 3
  35. // inside `*bar()`: D
  36. // outside: E
  38. console.log( "outside:", it.next( 4 ).value );
  39. // inside `*bar()`: 4
  40. // outside: F

Pay particular attention to the processing steps after the it.next(3) call:

  1. The 3 value is passed (through the yield-delegation in *bar()) into the waiting yield "C" expression inside of *foo().
  2. *foo() then calls return "D", but this value doesn’t get returned all the way back to the outside it.next(3) call.
  3. Instead, the "D" value is sent as the result of the waiting yield *foo() expression inside of *bar() — this yield-delegation expression has essentially been paused while all of *foo() was exhausted. So "D" ends up inside of *bar() for it to print out.
  4. yield "E" is called inside of *bar(), and the "E" value is yielded to the outside as the result of the it.next(3) call.

From the perspective of the external iterator (it), it doesn’t appear any differently between controlling the initial generator or a delegated one.

In fact, yield-delegation doesn’t even have to be directed to another generator; it can just be directed to a non-generator, general iterable. For example:

  1. function *bar() {
  2.     console.log( "inside `*bar()`:", yield "A" );
  4.     // `yield`-delegation to a non-generator!
  5.     console.log( "inside `*bar()`:", yield *[ "B", "C", "D" ] );
  7.     console.log( "inside `*bar()`:", yield "E" );
  9.     return "F";
  10. }
  12. var it = bar();
  14. console.log( "outside:", it.next().value );
  15. // outside: A
  17. console.log( "outside:", it.next( 1 ).value );
  18. // inside `*bar()`: 1
  19. // outside: B
  21. console.log( "outside:", it.next( 2 ).value );
  22. // outside: C
  24. console.log( "outside:", it.next( 3 ).value );
  25. // outside: D
  27. console.log( "outside:", it.next( 4 ).value );
  28. // inside `*bar()`: undefined
  29. // outside: E
  31. console.log( "outside:", it.next( 5 ).value );
  32. // inside `*bar()`: 5
  33. // outside: F

Notice the differences in where the messages were received/reported between this example and the one previous.

Most strikingly, the default array iterator doesn’t care about any messages sent in via next(..) calls, so the values 2, 3, and 4 are essentially ignored. Also, because that iterator has no explicit return value (unlike the previously used *foo()), the yield * expression gets an undefined when it finishes.

Exceptions Delegated, Too!

In the same way that yield-delegation transparently passes messages through in both directions, errors/exceptions also pass in both directions:

  1. function *foo() {
  2.     try {
  3.         yield "B";
  4.     }
  5.     catch (err) {
  6.         console.log( "error caught inside `*foo()`:", err );
  7.     }
  9.     yield "C";
  11.     throw "D";
  12. }
  14. function *bar() {
  15.     yield "A";
  17.     try {
  18.         yield *foo();
  19.     }
  20.     catch (err) {
  21.         console.log( "error caught inside `*bar()`:", err );
  22.     }
  24.     yield "E";
  26.     yield *baz();
  28.     // note: can't get here!
  29.     yield "G";
  30. }
  32. function *baz() {
  33.     throw "F";
  34. }
  36. var it = bar();
  38. console.log( "outside:", it.next().value );
  39. // outside: A
  41. console.log( "outside:", it.next( 1 ).value );
  42. // outside: B
  44. console.log( "outside:", it.throw( 2 ).value );
  45. // error caught inside `*foo()`: 2
  46. // outside: C
  48. console.log( "outside:", it.next( 3 ).value );
  49. // error caught inside `*bar()`: D
  50. // outside: E
  52. try {
  53.     console.log( "outside:", it.next( 4 ).value );
  54. }
  55. catch (err) {
  56.     console.log( "error caught outside:", err );
  57. }
  58. // error caught outside: F

Some things to note from this snippet:

  1. When we call it.throw(2), it sends the error message 2 into *bar(), which delegates that to *foo(), which then catches it and handles it gracefully. Then, the yield "C" sends "C" back out as the return value from the it.throw(2) call.
  2. The "D" value that’s next thrown from inside *foo() propagates out to *bar(), which catches it and handles it gracefully. Then the yield "E" sends "E" back out as the return value from the it.next(3) call.
  3. Next, the exception thrown from *baz() isn’t caught in *bar() — though we did catch it outside — so both *baz() and *bar() are set to a completed state. After this snippet, you would not be able to get the "G" value out with any subsequent next(..) call(s) — they will just return undefined for value.

Delegating Asynchrony

Let’s finally get back to our earlier yield-delegation example with the multiple sequential Ajax requests:

  1. function *foo() {
  2.     var r2 = yield request( "http://some.url.2" );
  3.     var r3 = yield request( "http://some.url.3/?v=" + r2 );
  5.     return r3;
  6. }
  8. function *bar() {
  9.     var r1 = yield request( "http://some.url.1" );
  11.     var r3 = yield *foo();
  13.     console.log( r3 );
  14. }
  16. run( bar );

Instead of calling yield run(foo) inside of *bar(), we just call yield *foo().

In the previous version of this example, the Promise mechanism (controlled by run(..)) was used to transport the value from return r3 in *foo() to the local variable r3 inside *bar(). Now, that value is just returned back directly via the yield * mechanics.

Otherwise, the behavior is pretty much identical.

Delegating “Recursion”

Of course, yield-delegation can keep following as many delegation steps as you wire up. You could even use yield-delegation for async-capable generator “recursion” — a generator yield-delegating to itself:

  1. function *foo(val) {
  2.     if (val > 1) {
  3.         // generator recursion
  4.         val = yield *foo( val - 1 );
  5.     }
  7.     return yield request( "http://some.url/?v=" + val );
  8. }
  10. function *bar() {
  11.     var r1 = yield *foo( 3 );
  12.     console.log( r1 );
  13. }
  15. run( bar );

Note: Our run(..) utility could have been called with run( foo, 3 ), because it supports additional parameters being passed along to the initialization of the generator. However, we used a parameter-free *bar() here to highlight the flexibility of yield *.

What processing steps follow from that code? Hang on, this is going to be quite intricate to describe in detail:

  1. run(bar) starts up the *bar() generator.
  2. foo(3) creates an iterator for *foo(..) and passes 3 as its val parameter.
  3. Because 3 > 1, foo(2) creates another iterator and passes in 2 as its val parameter.
  4. Because 2 > 1, foo(1) creates yet another iterator and passes in 1 as its val parameter.
  5. 1 > 1 is false, so we next call request(..) with the 1 value, and get a promise back for that first Ajax call.
  6. That promise is yielded out, which comes back to the *foo(2) generator instance.
  7. The yield * passes that promise back out to the *foo(3) generator instance. Another yield * passes the promise out to the *bar() generator instance. And yet again another yield * passes the promise out to the run(..) utility, which will wait on that promise (for the first Ajax request) to proceed.
  8. When the promise resolves, its fulfillment message is sent to resume *bar(), which passes through the yield * into the *foo(3) instance, which then passes through the yield * to the *foo(2) generator instance, which then passes through the yield * to the normal yield that’s waiting in the *foo(3) generator instance.
  9. That first call’s Ajax response is now immediately returned from the *foo(3) generator instance, which sends that value back as the result of the yield * expression in the *foo(2) instance, and assigned to its local val variable.
  10. Inside *foo(2), a second Ajax request is made with request(..), whose promise is yielded back to the *foo(1) instance, and then yield * propagates all the way out to run(..) (step 7 again). When the promise resolves, the second Ajax response propagates all the way back into the *foo(2) generator instance, and is assigned to its local val variable.
  11. Finally, the third Ajax request is made with request(..), its promise goes out to run(..), and then its resolution value comes all the way back, which is then returned so that it comes back to the waiting yield * expression in *bar().

Phew! A lot of crazy mental juggling, huh? You might want to read through that a few more times, and then go grab a snack to clear your head!