Serial and Parallel Experiments

Chaos Mesh Workflow offers two ways of scheduling experiments: serial and parallel. You can configure and schedule multiple experiments as needed.

  • If you want to schedule multiple chaos experiments in sequence, use serial nodes.
  • If you want to perform multiple chaos experiments simultaneously, use parallel nodes.

Chaos Mesh uses composite pattern when designing serial and parallel nodes. It can contain multiple nodes of different types and run the composite nodes in a specific mode. This also means that you can nest the serial and parallel nodes to achieve complicated scheduling.

Serial experiments

When you create templates in Workflow, use templateType: Serial to claim a serial node.

Another required field in serial nodes is children. Its type is []string and value is the name of other template. For example:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: Workflow
  3. metadata:
  4.   name: try-workflow-serial
  5. spec:
  6.   entry: serial-of-3-node
  7.   templates:
  8.     - name: serial-of-3-node
  9.       templateType: Serial
  10.       deadline: 240s
  11.       children:
  12.         - workflow-stress-chaos
  13.         - suspending
  14.         - workflow-network-chaos
  15.     - name: suspending
  16.       templateType: Suspend
  17.       deadline: 10s
  18.     - name: workflow-network-chaos
  19.       templateType: NetworkChaos
  20.       deadline: 20s
  21.       networkChaos:
  22.         direction: to
  23.         action: delay
  24.         mode: all
  25.         selector:
  26.           labelSelectors:
  27.             'app': 'hello-kubernetes'
  28.         delay:
  29.           latency: '90ms'
  30.           correlation: '25'
  31.           jitter: '90ms'
  32.     - name: workflow-stress-chaos
  33.       templateType: StressChaos
  34.       deadline: 20s
  35.       stressChaos:
  36.         mode: one
  37.         selector:
  38.           labelSelectors:
  39.             'app': 'hello-kubernetes'
  40.         stressors:
  41.           cpu:
  42.             workers: 1
  43.             load: 20
  44.             options: ['--cpu 1', '--timeout 600']

The above commands claims a serial node named serial-of-3-node. This means Chaos Mesh executes sequentially workflow-stress-chaos, suspending, and workflow-network-chaos. After all tasks are completed, serial nodes are marked as completed.

When Chaos Mesh executes the serial node, tasks claimed in children are run sequentially to ensure that only one task is running at the same time.

The deadline field in serial nodes is optional to limit the maximum duration of the entire serial process. Once this duration is running out, the sub-nodes are stopped and the nodes that are not executed yet will not be executed. If all sub-nodes finish their work before deadline time, serial nodes are immediately marked as completed and deadline is not affected.

Parallel experiments

When you create templates in Workflow, use templateType: Parallel to claim a parallel node.

Another required field in parallel nodes is children. Its type is []string and values are the names of other template. For example:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: Workflow
  3. metadata:
  4.   name: try-workflow-parallel
  5. spec:
  6.   entry: parallel-of-2-chaos
  7.   templates:
  8.     - name: parallel-of-2-chaos
  9.       templateType: Parallel
  10.       deadline: 240s
  11.       children:
  12.         - workflow-stress-chaos
  13.         - workflow-network-chaos
  14.     - name: workflow-network-chaos
  15.       templateType: NetworkChaos
  16.       deadline: 20s
  17.       networkChaos:
  18.         direction: to
  19.         action: delay
  20.         mode: all
  21.         selector:
  22.           labelSelectors:
  23.             'app': 'hello-kubernetes'
  24.         delay:
  25.           latency: '90ms'
  26.           correlation: '25'
  27.           jitter: '90ms'
  28.     - name: workflow-stress-chaos
  29.       templateType: StressChaos
  30.       deadline: 20s
  31.       stressChaos:
  32.         mode: one
  33.         selector:
  34.           labelSelectors:
  35.             'app': 'hello-kubernetes'
  36.         stressors:
  37.           cpu:
  38.             workers: 1
  39.             load: 20
  40.             options: ['--cpu 1', '--timeout 600']

The above commands claimed a parallel node named parallel-of-2-chaos. This means Chaos Mesh executes simultaneously workflow-stress-chaos and workflow-network-chaos. After all tasks are completed, parallel nodes are marked as completed.

When Chaos Mesh executes parallel nodes, all tasks claimed in children are executed simultaneously.

Similar to serial nodes, the optional field deadline is also available in parallel nodes to limit the maximum execution time of the entire parallel process. If this time is reached, the sub-nodes are stopped. If all sub-nodes finish their work before deadline time, parallel nodes are immediately marked as completed and deadline is not affected.

Create a workflow with serial or parallel nodes using Chaos Dashboard

Create serial nodes

Chaos Dashboard creates a predefined serial node called entry. Therefore, when creating a workflow with serial nodes using Chaos Dashboard, the workflow is created under entry by default.

Create Serial Node On Dashboard

Create parallel nodes

You can create a parallel node Parallel and create sub-nodes under Parallel.

Create Parallel Node on Dashboard

Nest serial and parallel nodes

You can create more complex processes by nesting serial and parallel nodes together.

Nested Serial And Parallel Node