Simulate Pod Faults

This document describes how to use Chaos Mesh to inject faults into Kubernetes Pod to simulate Pod or container faults. Chaos Dashboard and YAML files are provided to create PodChaos experiments.

PodChaos introduction

PodChaos is a fault type in Chaos Mesh. By creating a PodChaos experiment, you can simulate fault scenarios of the specified Pods or containers. Currently, PodChaos supports the following fault types:

• Pod Failure: injects fault into a specified Pod to make the Pod unavailable for a period of time.
• Pod Kill: kills a specified Pod.To ensure that the Pod can be successfully restarted, you need to configure ReplicaSet or similar mechanisms.
• Container Kill: kills the specified container in the target Pod.

Usage restrictions

Currently, Chaos Mesh only supports fault injection to certain types of Pod, such as Deployment, StatefulSet, and DaemonSet. Chaos Mesh does not support injecting faults into an independent Pod. An independent Pod means a Pod that is not bound to ReplicaSet or Deployment Pod.

Notes

Before creating PodChaos experiments, ensure the following:

• There is no Control Manager of Chaos Mesh running on the target Pod.
• If the fault type is Pod Kill, replicaSet or a similar mechanism is configured to ensure that Pod can restart automatically.

Create Experiments Using Chaos Dashboard

note" class="reference-link">note

Before create experiments using Chaos Dashboard, ensure the following:

• Chaos Dashboard is installed.
• If Chaos Dashboard is already installed, you can run kubectl port-forward to access Dashboard: bash kubectl port-forward -n chaos-testing svc/chaos-dashboard 2333:2333. Then you can enter http://localhost:2333 to access Chaos Dashboard.

1. Open Chaos Dashboard, and click NEW EXPERIMENT on the page to create a new experiment.

   

2. In the Choose a Target area, choose POD FAULT and select a specific behavior, such as POD FAILURE.

3. Fill out the experiment information, and specify the experiment scope and the scheduled experiment duration.

4. Submit the experiment information.

Create experiments using YAML configuration files

pod-failure example

1. Write the experiment configuration to the pod-failure.yaml file:

   apiVersion: chaos-mesh.org/v1alpha1
   kind: PodChaos
   metadata:
     name: pod-failure-example
     namespace: chaos-testing
   spec:
     action: pod-failure
     mode: one
     duration: '30s'
     selector:
       labelSelectors:
         'app.kubernetes.io/component': 'tikv'

   Based on this example, Chaos Mesh injects pod-failure into the specified Pod and makes the Pod unavailable for 30 seconds.

2. After the configuration file is prepared, use kubectl to create an experiment:

   kubectl apply -f ./pod-failure.yaml

pod-kill example

1. Write the experiment configuration to the pod-kill.yaml file:

   apiVersion: chaos-mesh.org/v1alpha1
   kind: PodChaos
   metadata:
     name: pod-kill-example
     namespace: chaos-testing
   spec:
     action: pod-kill
     mode: one
     selector:
       namespaces:
         - tidb-cluster-demo
       labelSelectors:
         'app.kubernetes.io/component': 'tikv'

   Based on this example, Chaos Mesh injects pod-kill into the specified Pod and kills the Pod once.

2. After the configuration file is prepared, use kubectl to create an experiment:

   kubectl apply -f ./pod-kill.yaml

container-kill example

1. Write the experiment configuration to the container-kill.yaml file:

   apiVersion: chaos-mesh.org/v1alpha1
   kind: PodChaos
   metadata:
     name: container-kill-example
     namespace: chaos-testing
   spec:
     action: container-kill
     mode: one
     containerNames: ['prometheus']
     selector:
       labelSelectors:
         'app.kubernetes.io/component': 'monitor'

   Based on this example, Chaos Mesh injects container-kill into the specified container and kills the container once.

2. After the configuration file is prepared, use kubectl to create an experiment:

   kubectl apply -f ./container-kill.yaml

Field description

The following table describes the fields in the YAML configuration file.