Chaos Mesh Workflow offers two ways of scheduling experiments: serial and parallel. You can configure and schedule multiple experiments as needed.

  • If you want to schedule multiple chaos experiments in sequence, use serial nodes.
  • If you want to perform multiple chaos experiments simultaneously, use parallel nodes.

Chaos Mesh uses composite pattern when designing serial and parallel nodes. It can contain multiple nodes of different types and run the composite nodes in a specific mode. This also means that you can nest the serial and parallel nodes to achieve complicated scheduling.

Serial experiments

When you create templates in Workflow, use templateType: Serial to claim a serial node.

Another required field in serial nodes is children. Its type is []string and value is the name of other template. For example:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: Workflow
  3. metadata:
  4.   name: try-workflow-serial
  5. spec:
  6.   entry: serial-of-3-node
  7.   templates:
  8.     - name: serial-of-3-node
  9.       templateType: Serial
  10.       duration: 240s
  11.       children:
  12.         - workflow-stress-chaos
  13.         - suspending
  14.         - workflow-network-chaos
  15.     - name: suspending
  16.       templateType: Suspend
  17.       duration: 10s
  18.     - name: workflow-network-chaos
  19.       templateType: NetworkChaos
  20.       duration: 20s
  21.       networkChaos:
  22.         direction: to
  23.         action: delay
  24.         mode: all
  25.         selector:
  26.           labelSelectors:
  27.             'app': 'hello-kubernetes'
  28.         delay:
  29.           latency: '90ms'
  30.           correlation: '25'
  31.           jitter: '90ms'
  32.     - name: workflow-stress-chaos
  33.       templateType: StressChaos
  34.       duration: 20s
  35.       stressChaos:
  36.         mode: one
  37.         selector:
  38.           labelSelectors:
  39.             'app': 'hello-kubernetes'
  40.         stressors:
  41.           cpu:
  42.             workers: 1
  43.             load: 20
  44.             options: ['--cpu 1', '--timeout 600']

The above commands claims a serial node named serial-of-3-node. This means Chaos Mesh executes sequentially workflow-stress-chaos, suspending, and workflow-network-chaos. After all tasks are completed, serial nodes are marked as completed.

When Chaos Mesh executes the serial node, tasks claimed in children are run sequentially to ensure that only one task is running at the same time.

The duration field in serial nodes is optional to limit the maximum duration of the entire serial process. Once this duration is running out, the sub-nodes are stopped and the nodes that are not executed yet will not be executed. If all sub-nodes finish their work before duration time, serial nodes are immediately marked as completed and duration is not affected.

Parallel experiments

When you create templates in Workflow, use templateType: Parallel to claim a parallel node.

Another required field in parallel nodes is children. Its type is []string and values are the names of other template. For example:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: Workflow
  3. metadata:
  4.   name: try-workflow-parallel
  5. spec:
  6.   entry: parallel-of-2-chaos
  7.   templates:
  8.     - name: parallel-of-2-chaos
  9.       templateType: Parallel
  10.       duration: 240s
  11.       children:
  12.         - workflow-stress-chaos
  13.         - workflow-network-chaos
  14.     - name: workflow-network-chaos
  15.       templateType: NetworkChaos
  16.       duration: 20s
  17.       networkChaos:
  18.         direction: to
  19.         action: delay
  20.         mode: all
  21.         selector:
  22.           labelSelectors:
  23.             'app': 'hello-kubernetes'
  24.         delay:
  25.           latency: '90ms'
  26.           correlation: '25'
  27.           jitter: '90ms'
  28.     - name: workflow-stress-chaos
  29.       templateType: StressChaos
  30.       duration: 20s
  31.       stressChaos:
  32.         mode: one
  33.         selector:
  34.           labelSelectors:
  35.             'app': 'hello-kubernetes'
  36.         stressors:
  37.           cpu:
  38.             workers: 1
  39.             load: 20
  40.             options: ['--cpu 1', '--timeout 600']

The above commands claimed a parallel node named parallel-of-2-chaos. This means Chaos Mesh executes simultaneously workflow-stress-chaos and workflow-network-chaos. After all tasks are completed, parallel nodes are marked as completed.

When Chaos Mesh executes parallel nodes, all tasks claimed in children are executed simultaneously.

Similar to serial nodes, the optional field duration is also available in parallel nodes to limit the maximum execution time of the entire parallel process. If this time is reached, the sub-nodes are stopped. If all sub-nodes finish their work before duration time, parallel nodes are immediately marked as completed and duration is not affected.