Initialize Env Infra Resources

This section will introduce how to initialize and destroy infrastructure of environment with KubeVela easily.

An Application development team usually needs to initialize some shared environment for users. An environment is a logical concept that represents a set of common infrastructure resources for Applications.

For example, a team usually wants two environments: one for development, and one for production.

In general, the infra resource types that can be initialized include the following types:

  1. One or more Kubernetes clusters. Different environments may need different sizes and versions of Kubernetes clusters. Environment initialization can also manage multiple clusters .

  2. Any type of Kubernetes custom resources (CRDs) and system plug-ins can be set up in environment initialization.

  3. All kinds of shared resources and services. For example. shared resources in microservices. These shared resources can be a microservice component, cloud database, cache, load balancer, API gateway, and so on.

  4. Various management policies and processes. An environment may have different global policies. The policy can be chaos test, security scan, SLO and son on; the process can be initializing a database table, registering an automatic discovery configuration, and so on.

KubeVela allows you to use different resources to initialize the environment.

You can use the Policy and Workflow in your Application. Note that there may be dependencies between initializations, we can use depends-on-app in workflow to do it.

The initialization of different environments has dependencies. Common resources can be separated as dependencies. In this way, reusable initialization modules can be formed.

For example, if both the test and develop environments rely on the same controllers, these controllers can be pulled out and initialized as separate environments, specifying dependency initialization in both the development and test environments.

If we want to use some CRD controller like OpenKruise in cluster, we can use Helm to initialize kruise.

We can directly use Application to initialize a kruise environment. The application below will deploy a kruise controller in cluster.

We have to enable FluxCD addon in cluster since we use Helm to deploy kruise. We can use depends-on-app to make sure addon-fluxcd is deployed before kruise, we also use apply-once policy here to make sure we’ll only apply the application once for initialization.

depends-on-app will check if the cluster has the application with name and namespace defines in properties. If the application exists, the next step will be executed after the application is running. If the application do not exists, KubeVela will check the ConfigMap with the same name, and read the config of the Application and apply to cluster. For more information, please refer to depends-on-app.

  1. cat <<EOF | vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5. name: kruise
  6. namespace: vela-system
  7. spec:
  8. components:
  9. - name: kruise
  10. type: helm
  11. properties:
  12. repoType: helm
  13. url: https://openkruise.github.io/charts/
  14. chart: kruise
  15. version: 1.2.0
  16. git:
  17. branch: master
  18. values:
  19. featureGates: PreDownloadImageForInPlaceUpdate=true
  20. policies:
  21. - name: apply-once
  22. type: apply-once
  23. properties:
  24. enable: true
  25. workflow:
  26. steps:
  27. - name: check-flux
  28. type: depends-on-app
  29. properties:
  30. name: addon-fluxcd
  31. namespace: vela-system
  32. - name: apply-kruise
  33. type: apply-component
  34. properties:
  35. component: kruise
  36. EOF

Check the application in cluster:

  1. vela status kruise -n vela-system

Expected Outcome

  1. About:
  2. Name: kruise
  3. Namespace: vela-system
  4. Created at: 2022-10-31 18:10:27 +0800 CST
  5. Status: running
  6. Workflow:
  7. mode: StepByStep-DAG
  8. finished: true
  9. Suspend: false
  10. Terminated: false
  11. Steps
  12. - id: l6apfsi5c2
  13. name: check-flux
  14. type: depends-on-app
  15. phase: succeeded
  16. - id: p2nqell47w
  17. name: apply-kruise
  18. type: apply-component
  19. phase: succeeded
  20. Services:
  21. - Name: kruise
  22. Cluster: local Namespace: vela-system
  23. Type: helm
  24. Healthy Fetch repository successfully, Create helm release successfully
  25. No trait applied

Kruise is running successfully! Then you can use kruise in your cluster. If you need to set up a new environment, the only thing you need to do is to apply the files like above.

Some Kubernetes native resources like ConfigMap/PVC are commonly used in the environment.

If you want to apply those resources before deploying your application, you can add an initialization workflow to your application.

KubeVela provides a built-in workflow step apply-object to fill in native Kubernetes resources. In this way, by filling in Kubernetes native resources, we can avoid writing redundant component definitions.

Apply the following application, it will initialize an environment with ConfigMap/PVC. There is two components in this application, the first one will write data to PVC, the second on will read the data from PVC:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4. name: server-with-pvc-and-cm
  5. namespace: default
  6. spec:
  7. components:
  8. - name: log-gen-worker
  9. type: worker
  10. properties:
  11. image: busybox
  12. cmd:
  13. - /bin/sh
  14. - -c
  15. - >
  16. i=0;
  17. while true;
  18. do
  19. echo "$i: $(date)" >> /test-pvc/date.log;
  20. i=$((i+1));
  21. sleep 1;
  22. done
  23. volumes:
  24. - name: "my-pvc"
  25. type: "pvc"
  26. mountPath: "/test-pvc"
  27. claimName: "my-claim"
  28. - name: "my-configmap"
  29. type: "configMap"
  30. mountPath: "/test-cm"
  31. cmName: "my-cm"
  32. items:
  33. - key: test-key
  34. path: test-key
  35. - name: log-read-worker
  36. type: worker
  37. properties:
  38. name: count-log
  39. image: busybox
  40. cmd:
  41. - /bin/sh
  42. - -c
  43. - 'tail -n+1 -f /test-pvc/date.log'
  44. volumes:
  45. - name: "my-pvc"
  46. type: "pvc"
  47. mountPath: "/test-pvc"
  48. claimName: "my-claim"
  49. - name: "my-configmap"
  50. type: "configMap"
  51. mountPath: "/test-cm"
  52. cmName: "my-cm"
  53. items:
  54. - key: test-key
  55. path: test-key
  56. policies:
  57. - name: my-policy
  58. properties:
  59. clusters:
  60. - local
  61. type: topology
  62. - name: apply-once
  63. type: apply-once
  64. properties:
  65. enable: true
  66. workflow:
  67. steps:
  68. - name: apply-pvc
  69. type: apply-object
  70. properties:
  71. value:
  72. apiVersion: v1
  73. kind: PersistentVolumeClaim
  74. metadata:
  75. name: my-claim
  76. namespace: default
  77. spec:
  78. accessModes:
  79. - ReadWriteOnce
  80. resources:
  81. requests:
  82. storage: 8Gi
  83. storageClassName: standard
  84. - name: apply-cm
  85. type: apply-object
  86. properties:
  87. value:
  88. apiVersion: v1
  89. kind: ConfigMap
  90. metadata:
  91. name: my-cm
  92. namespace: default
  93. data:
  94. test-key: test-value
  95. - name: deploy-comp
  96. properties:
  97. policies:
  98. - my-policy
  99. type: deploy

Check the PVC and ConfigMap in cluster:

  1. vela status server-with-pvc-and-cm --detail --tree

Expected Outcome

  1. CLUSTER NAMESPACE RESOURCE STATUS APPLY_TIME DETAIL
  2. local ─── default ─┬─ ConfigMap/my-cm updated 2022-10-31 18:00:52 Data: 1 Age: 57s
  3. ├─ PersistentVolumeClaim/my-claim updated 2022-10-31 10:00:52 Status: Bound Volume: pvc-b6f88ada-af98-468d-8cdd-31ca110c5e1a Capacity: 8Gi Access Modes: RWO StorageClass: standard Age: 57s
  4. ├─ Deployment/log-gen-worker updated 2022-10-31 10:00:52 Ready: 1/1 Up-to-date: 1 Available: 1 Age: 57s
  5. └─ Deployment/log-read-worker updated 2022-10-31 10:00:52 Ready: 1/1 Up-to-date: 1 Available: 1 Age: 57s

Check the application in cluster:

  1. vela status server-with-pvc-and-cm

Expected Outcome

  1. $ vela status server-with-pvc-and-cm
  2. About:
  3. Name: server-with-pvc-and-cm
  4. Namespace: default
  5. Created at: 2022-10-31 18:00:51 +0800 CST
  6. Status: running
  7. Workflow:
  8. mode: StepByStep-DAG
  9. finished: true
  10. Suspend: false
  11. Terminated: false
  12. Steps
  13. - id: xboizfjo28
  14. name: apply-pvc
  15. type: apply-object
  16. phase: succeeded
  17. - id: 4ngx25mrx8
  18. name: apply-cm
  19. type: apply-object
  20. phase: succeeded
  21. - id: 1gzzt3mfw1
  22. name: deploy-comp
  23. type: deploy
  24. phase: succeeded
  25. Services:
  26. - Name: log-gen-worker
  27. Cluster: local Namespace: default
  28. Type: worker
  29. Healthy Ready:1/1
  30. No trait applied
  31. - Name: log-read-worker
  32. Cluster: local Namespace: default
  33. Type: worker
  34. Healthy Ready:1/1
  35. No trait applied

Check the logs of the second component:

  1. vela logs server-with-pvc-and-cm --component log-read-worker

Expected Outcome

  1. + log-read-worker-7f4bc9d9b5-kb5l6 log-read-worker
  2. log-read-worker 2022-10-31T10:01:15.606903716Z 0: Mon Oct 31 10:01:13 UTC 2022
  3. log-read-worker 2022-10-31T10:01:15.606939383Z 1: Mon Oct 31 10:01:14 UTC 2022
  4. log-read-worker 2022-10-31T10:01:15.606941883Z 2: Mon Oct 31 10:01:15 UTC 2022
  5. log-read-worker 2022-10-31T10:01:16.607006425Z 3: Mon Oct 31 10:01:16 UTC 2022
  6. log-read-worker 2022-10-31T10:01:17.607184925Z 4: Mon Oct 31 10:01:17 UTC 2022
  7. log-read-worker 2022-10-31T10:01:18.607304426Z 5: Mon Oct 31 10:01:18 UTC 2022
  8. ...

We can see that both components is running. The two components share the same PVC and use the same ConfigMap.

As we have already modeled the environment as a KubeVela Application, we can destroy the environment easily by deleting the application.

  1. vela delete server-with-pvc-and-cm

Then the KubeVela controller will clean up all these resources.

Last updated on Aug 4, 2023 by Daniel Higuero