Develop a New Chaos

After preparing the development environment, let’s develop a new type of chaos, HelloWorldChaos, which only prints a “Hello World!” message to the log. Generally, to add a new chaos type for Chaos Mesh, you need to take the following steps:

  1. Define the schema type
  2. Register the CRD
  3. Register the handler for this chaos object
  4. Make the Docker image
  5. Run chaos

Define the schema type

To define the schema type for the new chaos object, add helloworldchaos_types.go in the api directory api/v1alpha1 and fill it with the following content:

  1. package v1alpha1
  4. import (
  5.     metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
  6. )
  9. // +kubebuilder:object:root=true
  10. // +chaos-mesh:base
  13. // HelloWorldChaos is the Schema for the helloworldchaos API
  14. type HelloWorldChaos struct {
  15.     metav1.TypeMeta   `json:",inline"`
  16.     metav1.ObjectMeta `json:"metadata,omitempty"`
  19.     Spec   HelloWorldChaosSpec   `json:"spec"`
  20.     Status HelloWorldChaosStatus `json:"status,omitempty"`
  21. }
  24. // HelloWorldChaosSpec is the content of the specification for a HelloWorldChaos
  25. type HelloWorldChaosSpec struct {
  26.     // Duration represents the duration of the chaos action
  27.     // +optional
  28.     Duration *string `json:"duration,omitempty"`
  31.     // Scheduler defines some schedule rules to control the running time of the chaos experiment about time.
  32.     // +optional
  33.     Scheduler *SchedulerSpec `json:"scheduler,omitempty"`
  34. }
  37. // HelloWorldChaosStatus represents the status of a HelloWorldChaos
  38. type HelloWorldChaosStatus struct {
  39.     ChaosStatus `json:",inline"`
  40. }

With this file added, the HelloWorldChaos schema type is defined. The structure of it can be described as the YAML file below:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: HelloWorldChaos
  3. metadata:
  4.   name: <name-of-this-resource>
  5.   namespace: <ns-of-this-resource>
  6. spec:
  7.   duration: <duration-of-every-action>
  8.   scheduler:
  9.     cron: <the-cron-job-definition-of-this-chaos>
  10. status:
  11.   phase: <phase-of-this-resource>
  12.   ...

make generate will generate boilerplate functions for it, which is needed to integrate the resource in the Chaos Mesh.

Register the CRD

The HelloWorldChaos object is a custom resource object in Kubernetes. This means you need to register the corresponding CRD in the Kubernetes API. Run make yaml, then the CRD will be generated in config/crd/bases/chaos-mesh.org_helloworldchaos.yaml. In order to combine all these YAML file into manifests/crd.yaml, modify kustomization.yaml by adding the corresponding line as shown below:

  1. resources:
  2.   - bases/chaos-mesh.org_podchaos.yaml
  3.   - bases/chaos-mesh.org_networkchaos.yaml
  4.   - bases/chaos-mesh.org_iochaos.yaml
  5.   - bases/chaos-mesh.org_helloworldchaos.yaml # this is the new line

Run make yaml again, and the definition of HelloWorldChaos will show in manifests/crd.yaml. You can check it through git diff

Register the handler for this chaos object

Create file controllers/helloworldchaos/types.go and fill it with following codes:

  1. package helloworldchaos
  4. import (
  5.     "context"
  8.     "k8s.io/apimachinery/pkg/runtime"
  9.     ctrl "sigs.k8s.io/controller-runtime"
  12.     "github.com/chaos-mesh/chaos-mesh/api/v1alpha1"
  13.     "github.com/chaos-mesh/chaos-mesh/pkg/router"
  14.     ctx "github.com/chaos-mesh/chaos-mesh/pkg/router/context"
  15.     end "github.com/chaos-mesh/chaos-mesh/pkg/router/endpoint"
  16. )
  19. type endpoint struct {
  20.     ctx.Context
  21. }
  24. func (e *endpoint) Apply(ctx context.Context, req ctrl.Request, chaos v1alpha1.InnerObject) error {
  25.     e.Log.Info("Hello World!")
  26.     return nil
  27. }
  30. func (e *endpoint) Recover(ctx context.Context, req ctrl.Request, chaos v1alpha1.InnerObject) error {
  31.     return nil
  32. }
  35. func (e *endpoint) Object() v1alpha1.InnerObject {
  36.     return &v1alpha1.HelloWorldChaos{}
  37. }
  40. func init() {
  41.     router.Register("helloworldchaos", &v1alpha1.HelloWorldChaos{}, func(obj runtime.Object) bool {
  42.         return true
  43.     }, func(ctx ctx.Context) end.Endpoint {
  44.         return &endpoint{
  45.             Context: ctx,
  46.         }
  47.     })
  48. }

We should also import github.com/chaos-mesh/chaos-mesh/controllers/helloworldchaos in the cmd/controller-manager/main.go, then it will register on the route table when the controller starts up.

Make the Docker image

Having the object successfully added, you can make a Docker image:

  1. make

Then push it to your registry:

  1. make docker-push

If your Kubernetes cluster is deployed by Kind, you need to load images to Kind:

  1. kind load docker-image localhost:5000/pingcap/chaos-mesh:latest
  2. kind load docker-image localhost:5000/pingcap/chaos-daemon:latest
  3. kind load docker-image localhost:5000/pingcap/chaos-dashboard:latest

Run chaos

You are almost there. In this step, you will pull the image and apply it for testing.

Before you pull any image for Chaos Mesh (using helm install or helm upgrade), modify values.yaml of the helm template to replace the default image with what you just pushed to your local registry.

In this case, the template uses pingcap/chaos-mesh:latest as the default target registry, so you need to replace it with the environment variable DOCKER_REGISTRY‘s value(default localhost:5000), as shown below:

  1. clusterScoped: true
  4. # Also see clusterScoped and controllerManager.serviceAccount
  5. rbac:
  6.   create: true
  9. controllerManager:
  10.   serviceAccount: chaos-controller-manager
  11.   ...
  12.   image: localhost:5000/pingcap/chaos-mesh:latest
  13.   ...
  14. chaosDaemon:
  15.   image: localhost:5000/pingcap/chaos-daemon:latest
  16.   ...
  17. dashboard:
  18.   image: localhost:5000/pingcap/chaos-dashboard:latest
  19.   ...

Now take the following steps to run chaos:

  1. Create namespace chaos-testing

    1. kubectl create namespace chaos-testing

  1. Get the related custom resource type for Chaos Mesh:

    1. kubectl apply -f manifests/
  1. You can see CRD `helloworldchaos` is created:
  3. ```
  4. customresourcedefinition.apiextensions.k8s.io/helloworldchaos.chaos-mesh.org created
  5. ```
  9. Now you can get the CRD using the command below:
  11. ```
  12. kubectl get crd helloworldchaos.chaos-mesh.org
  13. ```

  1. Install Chaos Mesh:

    • For helm 3.X

      1. helm install chaos-mesh helm/chaos-mesh --namespace=chaos-testing --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock
  1. -   For helm 2.X
  3.     ```
  4.     helm install helm/chaos-mesh --name=chaos-mesh --namespace=chaos-testing --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock
  5.     ```
  9. To verify your installation, get pods from the `chaos-testing` namespace:
  11. ```
  12. kubectl get pods --namespace chaos-testing -l app.kubernetes.io/instance=chaos-mesh
  13. ```
  17. > **Note:**
  18. > 
  19. > Arguments `--set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock` are used to to support network chaos on kind.

  1. Create chaos.yaml in any location with the lines below:

    1. apiVersion: chaos-mesh.org/v1alpha1
    2. kind: HelloWorldChaos
    3. metadata:
    4.   name: hello-world
    5.   namespace: chaos-testing
    6. spec: {}

  1. Apply the chaos:

    1. kubectl apply -f /path/to/chaos.yaml
  1. ```
  2. kubectl get HelloWorldChaos -n chaos-testing
  3. ```
  7. Now you should be able to check the `Hello World!` result in the log of of `chaos-controller-manager`:
  9. ```
  10. kubectl logs chaos-controller-manager-{pod-post-fix} -n chaos-testing
  11. ```
  15. The log is as follows:
  17. ```
  18. 2020-09-07T09:21:29.301Z        INFO    controllers.HelloWorldChaos     Hello World!    {"reconciler": "helloworldchaos"}
  19. 2020-09-07T09:21:29.308Z        DEBUG   controller-runtime.controller   Successfully Reconciled {"controller": "helloworldchaos", "request": "chaos-testing/hello-world"}
  20. ```
  24. > **Note:**
  25. > 
  26. > `{pod-post-fix}` is a random string generated by Kubernetes. You can check it by executing `kubectl get pod -n chaos-testing`.

Next steps

Congratulations! You have just added a chaos type for Chaos Mesh successfully. Let us know if you run into any issues during the process. If you feel like doing other types of contributions, refer to Add facilities to chaos daemon.