自定义运维特征

本节介绍如何自定义运维特征,为用户的组件增添任何需要的运维特征能力。开始这一部分之前,请确保你已经对核心概念 以及 如何管理模块定义有了基本的了解。

通过 Trait 生成资源的用法和 Component 基本类似,有如下场景:

  • 用于生成运维的资源对象,比如用于服务访问的 Ingress、Service,或者用于扩缩容的 HPA 等对象。
  • 用于上述运维对象的多种组合。
  • 用于对组件配置的补丁,如增加一个边车容器做日志收集等。

同样的,我们使用 vela def init命令来生成一个框架:

  1. vela def init my-route -t trait --desc "My ingress route trait." > myroute.cue

期望生成的内容如下:

  1. $ cat myroute.cue
  2. "my-route": {
  3. annotations: {}
  4. attributes: {
  5. appliesToWorkloads: []
  6. conflictsWith: []
  7. podDisruptive: false
  8. workloadRefPath: ""
  9. }
  10. description: "My ingress route trait."
  11. labels: {}
  12. type: "trait"
  13. }
  14. template: {
  15. patch: {}
  16. parameter: {}
  17. }

自定义运维特征 - 图1警告

在 vela CLI(<=1.4.2)的版本中有一个已知问题,vela def init 命令会生成一个错误的 definitionRef: "" 字段,这一行需要删除。

与组件定义有所不同,在用法上,你需要把所有的运维特征定义在 outputs 里(注意,不是 output),格式如下:

  1. outputs: {
  2. <unique-name>: {
  3. <template of trait resource structural data>
  4. }
  5. }

自定义运维特征 - 图2提示

Actually the CUE template of trait here will be evaluated with component CUE template in the same context, so the name can’t be conflict. That also explain why the output can’t be defined in trait.

我们下面使用一个 ingressService 组成一个称为 my-route 的运维特征作为示例讲解:

  1. "my-route": {
  2. annotations: {}
  3. attributes: {
  4. appliesToWorkloads: []
  5. conflictsWith: []
  6. podDisruptive: false
  7. workloadRefPath: ""
  8. }
  9. description: "My ingress route trait."
  10. labels: {}
  11. type: "trait"
  12. }
  13. template: {
  14. parameter: {
  15. domain: string
  16. http: [string]: int
  17. }
  18. // 我们可以在一个运维特征 CUE 模版定义多个 outputs
  19. outputs: service: {
  20. apiVersion: "v1"
  21. kind: "Service"
  22. spec: {
  23. selector:
  24. app: context.name
  25. ports: [
  26. for k, v in parameter.http {
  27. port: v
  28. targetPort: v
  29. },
  30. ]
  31. }
  32. }
  33. outputs: ingress: {
  34. apiVersion: "networking.k8s.io/v1beta1"
  35. kind: "Ingress"
  36. metadata:
  37. name: context.name
  38. spec: {
  39. rules: [{
  40. host: parameter.domain
  41. http: {
  42. paths: [
  43. for k, v in parameter.http {
  44. path: k
  45. backend: {
  46. serviceName: context.name
  47. servicePort: v
  48. }
  49. },
  50. ]
  51. }
  52. }]
  53. }
  54. }
  55. }

将这个运维特征通过如下命令部署到控制平面上:

  1. vela def apply myroute.cue

然后最终用户就立即可以发现并使用这个运维特征了,这个运维特征没有限制,可以作用于任意 Application

我们通过如下的 vela up 命令将它启动起来:

  1. cat <<EOF | vela up -f -
  2. apiVersion: core.oam.dev/v1beta1
  3. kind: Application
  4. metadata:
  5. name: testapp
  6. spec:
  7. components:
  8. - name: express-server
  9. type: webservice
  10. properties:
  11. cmd:
  12. - node
  13. - server.js
  14. image: oamdev/testapp:v1
  15. port: 8080
  16. traits:
  17. - type: my-route
  18. properties:
  19. domain: test.my.domain
  20. http:
  21. "/api": 8080
  22. EOF

然后 KubeVela 在服务端就会将其生成 Kubernetes 资源,通过 CUE 我们可以完成很多复杂的玩法。

你可以在 outputs 里定义 For 循环。

自定义运维特征 - 图3备注

注意在 For 循环里的 parameter 字段必须是 map 类型。

看看如下这个例子,在一个 TraitDefinition 对象里渲染多个 Service

  1. "expose": {
  2. type: "trait"
  3. }
  4. template: {
  5. parameter: {
  6. http: [string]: int
  7. }
  8. outputs: {
  9. for k, v in parameter.http {
  10. "\(k)": {
  11. apiVersion: "v1"
  12. kind: "Service"
  13. spec: {
  14. selector:
  15. app: context.name
  16. ports: [{
  17. port: v
  18. targetPort: v
  19. }]
  20. }
  21. }
  22. }
  23. }
  24. }

这个运维特征可以这样使用:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. metadata:
  4. name: testapp
  5. spec:
  6. components:
  7. - name: express-server
  8. type: webservice
  9. properties:
  10. ...
  11. traits:
  12. - type: expose
  13. properties:
  14. http:
  15. myservice1: 8080
  16. myservice2: 8081

TraitDefinition 对象可以发送 HTTP 请求并获取应答,让你可以通过关键字 processing 来渲染资源。

你可以在 processing.http 里定义 HTTP 请求的 method, url, body, headertrailer,然后返回的数据将被存储在 processing.output 中。

自定义运维特征 - 图4提示

请确保目标 HTTP 服务器返回的数据是 JSON 格式才能正确解析到 output 字段中。

接着,你就可以通过 patch 或者 output/outputs 里的 processing.output 来引用返回数据了。

下面是一个示例:

  1. "auth-service": {
  2. type: "trait"
  3. }
  4. template: {
  5. parameter: {
  6. serviceURL: string
  7. }
  8. processing: {
  9. output: {
  10. token?: string
  11. }
  12. // The target server will return a JSON data with `token` as key.
  13. http: {
  14. method: *"GET" | string
  15. url: parameter.serviceURL
  16. request: {
  17. body?: bytes
  18. header: {}
  19. trailer: {}
  20. }
  21. }
  22. }
  23. patch: {
  24. data: token: processing.output.token
  25. }
  26. }

在上面这个例子中,TraitDefinition 对象发送请求来获取 token 的数据,然后将这些数据补丁给组件实例。

TraitDefinition 对象可以读取特定 ComponentDefinition 对象生成的 API 资源(渲染自 outputoutputs)。

自定义运维特征 - 图5警告

Generally, KubeVela will ensure the component definitions are evaluated before its traits. But there’re a stage mechanism that allow trait be deployed before component.

具体来说,context.output 字段包含了所有渲染后的工作负载 API 资源,然后 context.outputs.<xx> 则包含渲染后的其它类型 API 资源。

下面是一个数据传递的例子:

  1. Let’s define a component definition myworker like below:
  1. "myworker": {
  2. attributes: workload: definition: {
  3. apiVersion: "apps/v1"
  4. kind: "Deployment"
  5. }
  6. type: "component"
  7. }
  8. template: {
  9. output: {
  10. apiVersion: "apps/v1"
  11. kind: "Deployment"
  12. spec: {
  13. selector: matchLabels: {
  14. "app.oam.dev/component": context.name
  15. }
  16. template: {
  17. metadata: labels: {
  18. "app.oam.dev/component": context.name
  19. }
  20. spec: {
  21. containers: [{
  22. name: context.name
  23. image: parameter.image
  24. ports: [{containerPort: parameter.port}]
  25. envFrom: [{
  26. configMapRef: name: context.name + "game-config"
  27. }]
  28. if parameter["cmd"] != _|_ {
  29. command: parameter.cmd
  30. }
  31. }]
  32. }
  33. }
  34. }
  35. }
  36. outputs: gameconfig: {
  37. apiVersion: "v1"
  38. kind: "ConfigMap"
  39. metadata: {
  40. name: context.name + "game-config"
  41. }
  42. data: {
  43. enemies: parameter.enemies
  44. lives: parameter.lives
  45. }
  46. }
  47. parameter: {
  48. // +usage=Which image would you like to use for your service
  49. // +short=i
  50. image: string
  51. // +usage=Commands to run in the container
  52. cmd?: [...string]
  53. lives: string
  54. enemies: string
  55. port: int
  56. }
  57. }
  1. Define a new myingress trait that read the port.
  1. "myingress": {
  2. type: "trait"
  3. attributes: {
  4. appliesToWorkloads: ["myworker"]
  5. }
  6. }
  7. template: {
  8. parameter: {
  9. domain: string
  10. path: string
  11. exposePort: int
  12. }
  13. // trait template can have multiple outputs in one trait
  14. outputs: service: {
  15. apiVersion: "v1"
  16. kind: "Service"
  17. spec: {
  18. selector:
  19. app: context.name
  20. ports: [{
  21. port: parameter.exposePort
  22. targetPort: context.output.spec.template.spec.containers[0].ports[0].containerPort
  23. }]
  24. }
  25. }
  26. outputs: ingress: {
  27. apiVersion: "networking.k8s.io/v1beta1"
  28. kind: "Ingress"
  29. metadata:
  30. name: context.name
  31. labels: config: context.outputs.gameconfig.data.enemies
  32. spec: {
  33. rules: [{
  34. host: parameter.domain
  35. http: {
  36. paths: [{
  37. path: parameter.path
  38. backend: {
  39. serviceName: context.name
  40. servicePort: parameter.exposePort
  41. }
  42. }]
  43. }
  44. }]
  45. }
  46. }
  47. }

在渲染 worker ComponentDefinition 时,具体发生了:

  1. 渲染的 Deployment 资源放在 context.output 中。
  2. 其它类型资源则放进 context.outputs.<xx> 中,同时 <xx> 是在特指 template.outputs 的唯一名字

因而,TraitDefinition 对象可以从 context 里读取渲染后的 API 资源(比如 context.outputs.gameconfig.data.enemies 这个字段)。

除了利用 Trait 生成资源以外,一个更高级的用法是对组件生成的参数做增补或修改。

什么场景下会使用这种功能?

  1. 组件由其他人定义,运维人员对参数做修改。
  2. 组件由第三方组织定义,我们不拥有修改能力(不维护),只在部署时使用。

针对上述场景,KubeVela 通过 patch 功能来支撑,因为 Patch 的能力针对 Trait 和 Workflow 均适用,我们通过这篇 Patch 文档统一介绍。

You can also define health check policy and status message when a trait deployed and tell the real status to end users.

The spec of health check is <trait-type-name>.attributes.status.healthPolicy, it’s similar to component definition.

If not defined, the health result will always be true, which means it will be marked as healthy immediately after resources applied to Kubernetes. You can define a CUE expression in it to notify if the trait is healthy or not.

The keyword in CUE is isHealth, the result of CUE expression must be bool type.

KubeVela runtime will evaluate the CUE expression periodically until it becomes healthy. Every time the controller will get all the Kubernetes resources and fill them into the context variables.

So the context will contain following information:

  1. context:{
  2. name: <component name>
  3. appName: <app name>
  4. outputs: {
  5. <resource1>: <K8s trait resource1>
  6. <resource2>: <K8s trait resource2>
  7. }
  8. }

The example of health check likes below:

  1. my-ingress: {
  2. type: "trait"
  3. ...
  4. attributes: {
  5. status: {
  6. healthPolicy: #"""
  7. isHealth: len(context.outputs.service.spec.clusterIP) > 0
  8. """#
  9. }
  10. }
  11. }

You can also use the parameter defined in the template like:

  1. mytrait: {
  2. type: "trait"
  3. ...
  4. attributes: {
  5. status: {
  6. healthPolicy: #"""
  7. isHealth: context.outputs."mytrait-\(parameter.name)".status.state == "Available"
  8. """#
  9. }
  10. }
  11. template: {
  12. parameter: {
  13. name: string
  14. }
  15. ...
  16. }

The health check result will be recorded into the corresponding trait in .status.services of Application resource.

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. status:
  4. ...
  5. services:
  6. - healthy: true
  7. ...
  8. name: myweb
  9. traits:
  10. - healthy: true
  11. type: my-ingress
  12. status: running

Please refer to this doc for more complete example.

The spec of custom status is <trait-type-name>.attributes.status.customStatus, it shares the same mechanism with the health check.

The keyword in CUE is message, the result of CUE expression must be string type.

Application CRD controller will evaluate the CUE expression after the health check succeed.

The example of custom status likes below:

  1. my-service: {
  2. type: "trait"
  3. ...
  4. attributes: {
  5. status: {
  6. customStatus: #"""
  7. if context.outputs.ingress.status.loadBalancer.ingress != _|_ {
  8. let igs = context.outputs.ingress.status.loadBalancer.ingress
  9. if igs[0].ip != _|_ {
  10. if igs[0].host != _|_ {
  11. message: "Visiting URL: " + context.outputs.ingress.spec.rules[0].host + ", IP: " + igs[0].ip
  12. }
  13. if igs[0].host == _|_ {
  14. message: "Host not specified, visit the cluster or load balancer in front of the cluster with IP: " + igs[0].ip
  15. }
  16. }
  17. }
  18. """#
  19. }
  20. }
  21. }

The message will be recorded into the corresponding trait in .status.services of Application resource.

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: Application
  3. status:
  4. ...
  5. services:
  6. - healthy: true
  7. ...
  8. name: myweb
  9. traits:
  10. - healthy: true
  11. message: 'Visiting URL: www.example.com, IP: 47.111.233.220'
  12. type: my-ingress
  13. status: running

Please refer to this doc for more complete example.

Context VariableDescriptionType
context.appNameThe app name corresponding to the current instance of the application.string
context.namespaceThe target namespace of the current resource is going to be deployed, it can be different with the namespace of application if overridden by some policies.string
context.clusterThe target cluster of the current resource is going to be deployed, it can be different with the namespace of application if overridden by some policies.string
context.appRevisionThe app version name corresponding to the current instance of the application.string
context.appRevisionNumThe app version number corresponding to the current instance of the application.int
context.nameThe component name corresponding to the current instance of the application.string
context.revisionThe version name of the current component instance.string
context.outputThe object structure after instantiation of current component.Object Map
context.outputs.<resourceName>Structure after instantiation of current component auxiliary resources.Object Map
context.workflowNameThe workflow name specified in annotation.string
context.publishVersionThe version of application instance specified in annotation.string
context.componentsThe object structure of components spec in this application.Object Map
context.appLabelsThe labels of the current application instance.Object Map
context.appAnnotationsThe annotations of the current application instance.Object Map
Context VariableDescriptionType
context.clusterVersion.majorThe major version of the runtime Kubernetes cluster.string
context.clusterVersion.gitVersionThe gitVersion of the runtime Kubernetes cluster.string
context.clusterVersion.platformThe platform information of the runtime Kubernetes cluster.string
context.clusterVersion.minorThe minor version of the runtime Kubernetes cluster.int

The cluster version context info can be used for graceful upgrade of definition. For example, you can define different API according to the cluster version.

  1. outputs: ingress: {
  2. if context.clusterVersion.minor < 19 {
  3. apiVersion: "networking.k8s.io/v1beta1"
  4. }
  5. if context.clusterVersion.minor >= 19 {
  6. apiVersion: "networking.k8s.io/v1"
  7. }
  8. kind: "Ingress"
  9. }

Or use string contain pattern for this usage:

  1. import "strings"
  2. if strings.Contains(context.clusterVersion.gitVersion, "k3s") {
  3. provider: "k3s"
  4. }
  5. if strings.Contains(context.clusterVersion.gitVersion, "aliyun") {
  6. provider: "aliyun"
  7. }

KubeVela is fully programmable via CUE, while it leverage Kubernetes as control plane and align with the API in yaml. As a result, the CUE definition will be converted as Kubernetes API when applied into cluster.

The trait definition will be in the following API format:

  1. apiVersion: core.oam.dev/v1beta1
  2. kind: TraitDefinition
  3. metadata:
  4. name: <TraitDefinition name>
  5. annotations:
  6. definition.oam.dev/description: <function description>
  7. spec:
  8. definition:
  9. apiVersion: <corresponding Kubernetes resource group>
  10. kind: <corresponding Kubernetes resource type>
  11. workloadRefPath: <The path to the reference field of the Workload object in the Trait>
  12. podDisruptive: <whether the parameter update of Trait cause the underlying resource (pod) to restart>
  13. manageWorkload: <Whether the workload is managed by this Trait>
  14. skipRevisionAffect: <Whether this Trait is not included in the calculation of version changes>
  15. appliesToWorkloads:
  16. - <Workload that TraitDefinition can adapt to>
  17. conflictsWith:
  18. - <other Traits that conflict with this><>
  19. revisionEnabled: <whether Trait is aware of changes in component version>
  20. controlPlaneOnly: <Whether resources generated by trait are dispatched to the hubcluster (local)>
  21. schematic: # Abstract
  22. cue: # There are many abstracts
  23. template: <CUE format template>

You can check the detail of this format here.

You can check the following resources for more examples:

Last updated on 2023年8月4日 by Daniel Higuero