Develop a New Chaos

After preparing the development environment, let’s develop a new type of chaos, HelloWorldChaos, which only prints a “Hello World!” message to the log. Generally, to add a new chaos type for Chaos Mesh, you need to take the following steps:

  1. Define the schema type
  2. Register the CRD
  3. Register the handler for this chaos object
  4. Make the Docker image
  5. Run chaos

Define the schema type

To define the schema type for the new chaos object, add helloworldchaos_types.go in the api directory api/v1alpha1 and fill it with the following content:

  1. package v1alpha1
  2. import (
  3. metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
  4. )
  5. // +kubebuilder:object:root=true
  6. // +chaos-mesh:base
  7. // HelloWorldChaos is the Schema for the helloworldchaos API
  8. type HelloWorldChaos struct {
  9. metav1.TypeMeta `json:",inline"`
  10. metav1.ObjectMeta `json:"metadata,omitempty"`
  11. Spec HelloWorldChaosSpec `json:"spec"`
  12. Status HelloWorldChaosStatus `json:"status,omitempty"`
  13. }
  14. // HelloWorldChaosSpec is the content of the specification for a HelloWorldChaos
  15. type HelloWorldChaosSpec struct {
  16. // Duration represents the duration of the chaos action
  17. // +optional
  18. Duration *string `json:"duration,omitempty"`
  19. // Scheduler defines some schedule rules to control the running time of the chaos experiment about time.
  20. // +optional
  21. Scheduler *SchedulerSpec `json:"scheduler,omitempty"`
  22. }
  23. // HelloWorldChaosStatus represents the status of a HelloWorldChaos
  24. type HelloWorldChaosStatus struct {
  25. ChaosStatus `json:",inline"`
  26. }

With this file added, the HelloWorldChaos schema type is defined. The structure of it can be described as the YAML file below:

  1. apiVersion: chaos-mesh.org/v1alpha1
  2. kind: HelloWorldChaos
  3. metadata:
  4. name: <name-of-this-resource>
  5. namespace: <ns-of-this-resource>
  6. spec:
  7. duration: <duration-of-every-action>
  8. scheduler:
  9. cron: <the-cron-job-definition-of-this-chaos>
  10. status:
  11. phase: <phase-of-this-resource>
  12. ...

make generate will generate boilerplate functions for it, which is needed to integrate the resource in the Chaos Mesh.

Register the CRD

The HelloWorldChaos object is a custom resource object in Kubernetes. This means you need to register the corresponding CRD in the Kubernetes API. Run make yaml, then the CRD will be generated in config/crd/bases/chaos-mesh.org_helloworldchaos.yaml. In order to combine all these YAML file into manifests/crd.yaml, modify kustomization.yaml by adding the corresponding line as shown below:

  1. resources:
  2. - bases/chaos-mesh.org_podchaos.yaml
  3. - bases/chaos-mesh.org_networkchaos.yaml
  4. - bases/chaos-mesh.org_iochaos.yaml
  5. - bases/chaos-mesh.org_helloworldchaos.yaml # this is the new line

Run make yaml again, and the definition of HelloWorldChaos will show in manifests/crd.yaml. You can check it through git diff

Register the handler for this chaos object

Create file controllers/helloworldchaos/types.go and fill it with following codes:

  1. package helloworldchaos
  2. import (
  3. "context"
  4. "k8s.io/apimachinery/pkg/runtime"
  5. ctrl "sigs.k8s.io/controller-runtime"
  6. "github.com/chaos-mesh/chaos-mesh/api/v1alpha1"
  7. "github.com/chaos-mesh/chaos-mesh/pkg/router"
  8. ctx "github.com/chaos-mesh/chaos-mesh/pkg/router/context"
  9. end "github.com/chaos-mesh/chaos-mesh/pkg/router/endpoint"
  10. )
  11. type endpoint struct {
  12. ctx.Context
  13. }
  14. func (e *endpoint) Apply(ctx context.Context, req ctrl.Request, chaos v1alpha1.InnerObject) error {
  15. e.Log.Info("Hello World!")
  16. return nil
  17. }
  18. func (e *endpoint) Recover(ctx context.Context, req ctrl.Request, chaos v1alpha1.InnerObject) error {
  19. return nil
  20. }
  21. func (e *endpoint) Object() v1alpha1.InnerObject {
  22. return &v1alpha1.HelloWorldChaos{}
  23. }
  24. func init() {
  25. router.Register("helloworldchaos", &v1alpha1.HelloWorldChaos{}, func(obj runtime.Object) bool {
  26. return true
  27. }, func(ctx ctx.Context) end.Endpoint {
  28. return &endpoint{
  29. Context: ctx,
  30. }
  31. })
  32. }

We should also import github.com/chaos-mesh/chaos-mesh/controllers/helloworldchaos in the cmd/controller-manager/main.go, then it will register on the route table when the controller starts up.

Make the Docker image

Having the object successfully added, you can make a Docker image:

  1. make

Then push it to your registry:

  1. make docker-push

If your Kubernetes cluster is deployed by Kind, you need to load images to Kind:

  1. kind load docker-image localhost:5000/pingcap/chaos-mesh:latest
  2. kind load docker-image localhost:5000/pingcap/chaos-daemon:latest
  3. kind load docker-image localhost:5000/pingcap/chaos-dashboard:latest

Run chaos

You are almost there. In this step, you will pull the image and apply it for testing.

Before you pull any image for Chaos Mesh (using helm install or helm upgrade), modify values.yaml of the helm template to replace the default image with what you just pushed to your local registry.

In this case, the template uses pingcap/chaos-mesh:latest as the default target registry, so you need to replace it with the environment variable DOCKER_REGISTRY‘s value(default localhost:5000), as shown below:

  1. clusterScoped: true
  2. # Also see clusterScoped and controllerManager.serviceAccount
  3. rbac:
  4. create: true
  5. controllerManager:
  6. serviceAccount: chaos-controller-manager
  7. ...
  8. image: localhost:5000/pingcap/chaos-mesh:latest
  9. ...
  10. chaosDaemon:
  11. image: localhost:5000/pingcap/chaos-daemon:latest
  12. ...
  13. dashboard:
  14. image: localhost:5000/pingcap/chaos-dashboard:latest
  15. ...

Now take the following steps to run chaos:

  1. Create namespace chaos-testing

    1. kubectl create namespace chaos-testing
  2. Get the related custom resource type for Chaos Mesh:

    1. kubectl apply -f manifests/

    You can see CRD helloworldchaos is created:

    1. customresourcedefinition.apiextensions.k8s.io/helloworldchaos.chaos-mesh.org created

    Now you can get the CRD using the command below:

    1. kubectl get crd helloworldchaos.chaos-mesh.org
  3. Install Chaos Mesh:

    • For helm 3.X

      1. helm install chaos-mesh helm/chaos-mesh --namespace=chaos-testing --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock
    • For helm 2.X

      1. helm install helm/chaos-mesh --name=chaos-mesh --namespace=chaos-testing --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock

    To verify your installation, get pods from the chaos-testing namespace:

    1. kubectl get pods --namespace chaos-testing -l app.kubernetes.io/instance=chaos-mesh

    Note:

    Arguments --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock are used to to support network chaos on kind.

  4. Create chaos.yaml in any location with the lines below:

    1. apiVersion: chaos-mesh.org/v1alpha1
    2. kind: HelloWorldChaos
    3. metadata:
    4. name: hello-world
    5. namespace: chaos-testing
    6. spec: {}
  5. Apply the chaos:

    1. kubectl apply -f /path/to/chaos.yaml
    1. kubectl get HelloWorldChaos -n chaos-testing

    Now you should be able to check the Hello World! result in the log of of chaos-controller-manager:

    1. kubectl logs chaos-controller-manager-{pod-post-fix} -n chaos-testing

    The log is as follows:

    1. 2020-09-07T09:21:29.301Z INFO controllers.HelloWorldChaos Hello World! {"reconciler": "helloworldchaos"}
    2. 2020-09-07T09:21:29.308Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "helloworldchaos", "request": "chaos-testing/hello-world"}

    Note:

    {pod-post-fix} is a random string generated by Kubernetes. You can check it by executing kubectl get pod -n chaos-testing.

Next steps

Congratulations! You have just added a chaos type for Chaos Mesh successfully. Let us know if you run into any issues during the process. If you feel like doing other types of contributions, refer to Add facilities to chaos daemon.