Gradually rolling out latest Revisions

If your traffic configuration points to a Configuration target, rather than revision target, it means that when a new Revision is created and ready 100% of that target’s traffic will be immediately shifted to the new revision, which might not be ready to accept that scale with a single pod and with cold starts taking some time it is possible to end up in a situation where a lot of requests are backed up either at QP or Activator and after a while they might expire or QP might outright reject the requests.

To mitigate this problem Knative as of 0.20 release Knative provides users with a possibility to gradually shift the traffic to the latest revision. This is governed by a single parameter which denotes rollout-duration.

The affected Configuration targets will be rolled out to 1% of traffic first and then in equal incremental steps for the rest of the assigned traffic. Note, that the rollout is purely time based and does not interact with the Autoscaling subsystem.

This feature is available to untagged and tagged traffic targets configured for both Kservices and Kservice-less Routes.

Configuring gradual Rollout

This value currently can be configured on the cluster level (starting v0.20) via a setting in the config-network ConfigMap or per Kservice or Route using an annotation (staring v.0.21).

  1. apiVersion: serving.knative.dev/v1
  2. kind: Service
  3. metadata:
  4.   name: helloworld-go
  5.   namespace: default
  6.   annotations:
  7.     serving.knative.dev/rolloutDuration: "380s"
  8. ...
  1. apiVersion: v1
  2. kind: ConfigMap
  3. metadata:
  4.  name: config-network
  5.  namespace: knative-serving
  6. data:
  7.   rolloutDuration: "380s"  # Value in seconds.
  1. apiVersion: operator.knative.dev/v1alpha1
  2. kind: KnativeServing
  3. metadata:
  4.   name: knative-serving
  5. spec:
  6.   config:
  7.     network:
  8.        rolloutDuration: "380s"

Route Status updates

During the rollout the system will update the Route and Kservice status. Both traffic and conditions status fields will be affected.

For example, a possible rollout of the following traffic configuration

  1. traffic:
  2. - percent: 55
  3.   configurationName: config # Pinned to latest ready Revision
  4. - percent: 45
  5.   revisionName: config-00005 # Pinned to a specific Revision.

would be (if inspecting the route status):

  1. traffic:
  2. - percent: 54
  3.   revisionName: config-00008
  4. - percent: 1
  5.   revisionName: config-00009
  6. - percent: 45
  7.   revisionName: config-00005 # Pinned to a specific Revision.

and then, presuming steps of 18%:

  1. traffic:
  2. - percent: 36
  3.   revisionName: config-00008
  4. - percent: 19
  5.   revisionName: config-00009
  6. - percent: 45
  7.   revisionName: config-00005 # Pinned to a specific Revision.

and so on until final state is achieved:

  1. traffic:
  2. - percent: 55
  3.   revisionName: config-00009
  4. - percent: 45
  5.   revisionName: config-00005 # Pinned to a specific Revision.

During the rollout the Route and (Kservice, if present) status conditions will be the following:

  1. ...
  2. status:
  3.   conditions:
  4.   ...
  5.   - lastTransitionTime: "..."
  6.     message: A gradual rollout of the latest revision(s) is in progress.
  7.     reason: RolloutInProgress
  8.     status: Unknown
  9.     type: Ready
  10. ...

Multiple Rollouts

If a new revision is created while the rollout is in progress then the system would start shifting the traffic immediately to the newest revision and it will drain the incomplete rollouts from newest to the oldest.